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Abstract 
In this technical report we document the development and piloting of the Learning Receptiveness 
Assessment (LRA) Greenhouse for Prekindergarten. This technological tool was designed to 
prevent reading disabilities by supporting effective assessment-guided instructional decision- 
making by Preschool teachers. The LRA Greenhouse comprises a tablet-based formative 
assessment and app-enabled research-based intentional teaching activity plans, monitoring 
checklist, and reports; it also includes a support website containing printable curriculum 
materials, online training modules, and teacher resources. The LRA measures emergent pre- 
reading skill levels, classroom behavior regulation, and working memory processing capacity in 
the fall, winter, and spring to provide teachers with multi-dimensional information for addressing 
children’s “high priority needs” in time for Kindergarten. Activity plans focus on building 
children’s literacy and behavior regulation for managing social-emotional and task engagement 
experiences, which are crucial to adequately develop during the Prekindergarten-Kindergarten 
transition as children adjust to classroom-based learning. We share our development process, 
descriptions and screenshots of wrap-around supports and features, as well as findings from two 
pilot studies conducted across 12 classrooms. Our goal is to present lessons learned from 
developing a technological tool intended to feasibly and sustainably “bridge a gap” in reading 
disabilities prevention between early childhood and K-12 schools, and preliminary evidence of 


positive impact for both Prekindergarten children and their teachers. 


Introduction 

The LRA Greenhouse grew from work developing the Learning Receptiveness 
Assessment (LRA), a tablet-based assessment designed to quickly, reliably, and easily identify 
Prekindergarten children at risk for experiencing persistent reading disabilities (RD). We aimed 
to extend prior school-based research that has shown how reading difficulties can be 
appropriately identified and intervened with before they become disabling (e.g., Chard, et al., 
2008; Compton, et al., 2012). However, similar approaches with younger children have yielded 
mixed results (e.g., Bailet, et al., 2009; Greenwood et al., 2013; Hindson, et al., 2005; 
VanDerHeyden, et al., 2007), suggesting there are unaddressed underlying issues. Through our 
LRA Greenhouse development work, we indeed discovered more deeply nuanced obstacles, 
which we discuss (along with the solutions we tried) in this technical report. Our hope to is to 
help advance the development of effective evidence-informed RD prevention tools for use in 
Prekindergarten by sharing our lessons learned and promising findings. 
Learning Across the Kindergarten Transition 

The broader Kindergarten transition (from Prekindergarten through the end of 
Kindergarten) entails an important and complex shift between home and formal schooling that 
shapes future learning success (Rimm-Kaufman & Pianta, 2000). This critical adjustment period 
involves more than a change in environment. Children must adapt to changes in interactions, 
contexts, routines, and connections; how successfully children can adapt influences how well 
they will respond to classroom instruction. “Learning” in a classroom is a fundamentally 
different enterprise than at home, with very different learning goals and ecologies (even when, 


for example, there is a shared focus on reading development). Together, Prekindergarten and 


Kindergarten classroom experiences frame how children transform into student learners, but 
their philosophically different approaches to learning create a transformation gap that can 
impede smooth transitions. 

For example, although Kindergarten signals a departure from early childhood and arrival 
into formal schooling, classroom learning is aligned with school expectations that have become 
increasingly skill-based and academic-focused (Bassok & Latham, 2017). The primary grades 
(Kindergarten through Second grade) are now a critical period for establishing timely reading 
development, thereby raising the importance of early reading difficulties identification to 
prevent the emergence of later, specific and disabling reading problems. In contrast, 
Prekindergarten classroom learning is aligned with home expectations because of their location 
in early childhood centers where children, as young as infants, are also served. This context 
directs their classroom focus and aims to educate the “whole child”, instead of the more skills- 
oriented (school-aligned), “pre-reading child”. Consequently, developing children’s literacy in 
Prekindergarten is an important, but not central, priority. Prekindergarten teachers focus on 
providing well-rounded classroom learning experiences that promote social, emotional, 
physical and health, cognitive, and general learning competencies, much like families do at 
home (NAEYC, 2020). However, increased concerns about children’s readiness to successfully 
learn in school highlights the pressing need for more coordinated strategies and enhanced 
alignment across the Prekindergarten-Kindergarten transition, to strengthen children’s learning 


outcomes (Atkins-Burnett, 2007; Wesley & Buysse, 2003). 


Strengthening Emergent Literacy Learning and RD Prevention 

Despite important differences in the ways that children learn across the Kindergarten 
transition, we believe that Prekindergarten is a particularly fruitful time for preventing RD, 
before reading development problems become disabling, precisely because it bridges home 
and school “learning”. Two years (Prekindergarten + Kindergarten) of coordinated support for 
developing robust reading skills may significantly reduce the incidence of reading difficulties 
experienced in school. More specifically, Prekindergarten is uniquely positioned to “level the 
playing field” as an approach to preventing RD prior to schooling—to help disentangle 
opportunity-to-learn disadvantages from “true” learning disabilities, a complexity that has long 
plagued the field (e.g., Algozzine, et al., 1982; Compton, et al., 2012; Fletcher, et al., 2007; 
Kavale & Forness, 2000; Lyon, 1994; Olson & Mealor, 1981; Swanson, 1991). Through the 
initiation of intentional, evidence-informed Prekindergarten strategies for addressing early 
learning difficulties as all children learn how to learn (Rimm-Kaufman & Pianta, 2000), those 
most vulnerable to experiencing RD can receive the additional and cohesive literacy learning 
support necessary for adequate prevention. 

Structural Challenges. However, important differences in how early childhood is 
structured, compared to K-12 schools, create challenges to effectively creating balanced 
alignment and coordination between Prekindergarten and Kindergarten approaches to reading 
development. Although increased attention has been drawn to children’s “readiness for 
Kindergarten”, current Prekindergarten options comprise a mix of state-funded, for-profit, and 
non-profit programs that exacerbate existing inequities in children’s early learning experiences. 


This hodge-podge of learning opportunities, in addition to at-home (non-classroom) care 


options, hamper the early identification of learning difficulties in Kindergarten: There is too 


|” 


much disadvantage “noise” to capture a true “signal” of persistent learning difficulties. 

Because children differentially transition to Kindergarten, more than just their skill levels 
are at play—varying abilities to receive instruction dramatically impact their academic 
performance. Thus, accurate and consistently measured early learning skills and their key 
contributors (e.g., learning-related behaviors, classroom engagement, executive functioning; 
McClelland & Morrison, 2003; Reilly & Downer, 2019; Williford et al., 2013), over time, are 
needed to clarify “high priority” learning differences across the diversity of Prekindergarten 
options (and effectively assist with disentangling “noisy” disadvantage). In doing so, a change to 
early childhood uses of assessment, beyond the typical diagnosis of developmental disabilities 
or meeting of program accountability requirements, is necessary. More specifically, adopting 
assessment use for guiding instructional practice, to precisely inform how children’s learning 
needs are supported, would elevate current intentional approaches to Prekindergarten 
teaching to a more strategic level for preventing reading difficulties. 

The LRA Greenhouse approach described in this Technical Report emphasizes the 
creation of an educational technological tool developed to enhance children’s literacy 
development by helping Prekindergarten practices “reach up” to Kindergarten practices to 
build a more seamless pathway to reading success and prevent RD (in contrast to “push down” 
Kindergarten practices into Prekindergarten). That is, we began our work on this project 
knowledgeable about effective school-based practices while acknowledging chief assets within 
early childhood systems. Our team sought to utilize the strengths of both approaches to 


learning to erect a Prekindergarten-Kindergarten transition “bridge”, by identifying and 


adjusting evidence-informed strategies known to work in Kindergarten and working closely with 
Prekindergarten classrooms to build sufficient scaffolded supports for enabling a feasible fit 
within existing ecologies. Below, we describe key Prekindergarten and Kindergarten inequities 
that influenced the types of supports developed for the LRA Greenhouse, to help better “level” 
the Prekindergarten-Kindergarten playing fields. 

Early Childhood Education Inequities. System-wide supports for ensuring high-quality 
learning between Kindergarten and Prekindergarten early childhood systems vary greatly. For 
example, Kindergarten teachers benefit from a long history of state-wide standards that outline 
expectations for high-quality instruction and learning success, and full administrative funding 
for curriculum and professional development. In contrast, within the United States, only 44 of 
50 states have adopted early childhood learning standards, and 27 states have policies that 
require paid professional development (Friedman-Krauss, et al., 2019). In addition, unlike the 
requirement that Kindergarten teachers maintain a teaching license, only 50% of states require 
Preschool teachers to minimally hold a bachelor’s degree and training certification (Friedman- 
Krauss, et al., 2019). Similarly, just half of states have “approved” or “recommended” 
curriculum to support widespread uses of effective instructional practices (Friedman-Krauss, et 
al., 2019). Thus, Preschool teachers face a distinct disadvantage in acquiring the knowledge, 
skills, and resources necessary to maintain high-quality instruction in the absence of concrete 
standards-based expectations, equal supports for professional development, basic 
requirements for certified training, and a state-wide consensus on recommended curricula and 


practices. 


Without clear early learning standards and formalized curricula to align what children 
should know by the end of the Prekindergarten year, adequately preparing al/ children to start 
school is a formidable challenge given the diversity of children’s experiences prior to 
Kindergarten. For example, among the nation’s nearly 2.4 million children, about half receive 
Prekindergarten classroom experience outside of their homes (Snyder et al., 2019): 48.7% 
attend centers, 19.5% receive care in either a relative’s home or another (non-relative) home, 
and 2.5% stay home. To promote successful transitions into and during Kindergarten, not only 
must curricula and instructional practices be flexible enough to serve diverse Prekindergarten 
experiences, but they must be linked to shared understandings and expectations for learning. 
Kindergarten classrooms benefit from district-centralized support for understanding the impact 
of standards on teaching and learning, whereas many Prekindergarten classrooms do not; 
consequently, idiosyncratic learning goals are often implemented across different classrooms 
(even within the same center). Thus, a fundamental aspect of RD prevention efforts during the 
Kindergarten transition must include building a shared pathway for developing reading 
readiness for Kindergarten despite diverse pre-kindergarten experiences. 

The lack of coordinated and systemic support for Prekindergarten children’s and 
teachers’ learning impacts: 

(a) teacher knowledge about what and how to prevent early learning difficulties, 

(b) programming quality for ensuring children’s adequate preparation for school, and 

(c) retention of providers committed to contributing to children’s incremental 

development beyond “day care” (e.g., high turn-over rates exist in Preschool 


classrooms; Whitebook & Sakai, 2003). 


Therefore, Prekindergarten RD prevention efforts must be more than strategic producers of 
children who are “reading-ready”, they must also enhance the quality of preschool 
programming more generally. In other words, to sustainably impact practice, RD prevention 
efforts must be feasible and flexible enough to enable effective implementation under varied 
and fluctuating conditions, and without great financial or time costs. 

Intervention as Prevention Solutions. For more than two decades, researchers have 
documented the pivotal role that Prekindergarten literacy skill development plays on later 
reading achievement in school (Lonigan, et al., 2000; Lundberg, et al., 1988; Storch & 
Whitehurst, 2002). In particular, skill levels measured during Preschool remain stable into the 
primary grades (Cabell, et al., 2011; Storch & Whitehurst, 2002). Therefore, we believed that a 
two-pronged approach to “learning” during the Kindergarten transition was needed, consisting 
of supports for children’s literacy development for directly preventing RD and equal focus on 
supporting Prekindergarten teachers’ development as active caretakers capable of ensuring 
solid literacy foundations in time for school. We believed that the skills and strategies teachers 
would learn needed to be doable, sustainable, and practice-shifting (in ways that more closely 
align, but do not exactly match, with the reading instruction children receive in Kindergarten). 

Children may exhibit weak literacy-related skills for a variety of reasons: disadvantages 
in the home, processing delays, behavioral difficulties that hinder learning, and/or inexperience 
(i.e., lack of opportunity to learn them). Studies of emergent reader skill profiles have revealed 
substantial heterogeneity in the performance patterns of children at risk for persistent RD 
(Cabell, et al., 2013; Ozernov-Palchik, et al., 2017). Because structured reading skill 


development occurs during the first year of school within the United States, Kindergarten 


teachers are faced with an inconceivable challenge when wide disparities in receptiveness for 
learning exist. The failure to adequately and efficiently build key skills, particularly in 
Kindergarten, can have long-lasting, negatively cascading effects (Bast & Reitsma, 1998; de Jong 
& ven der Leij, 2003; Saez, et al., 2016). Early intervention can help prevent problems from 
becoming RD (e.g., Al Otaiba, Connor, et al., 2008; Justice, et al., 2003; Torgesen, et al., 1999), 
but the consistency of instructional quality is an important mitigating factor that must be 
considered (Greenwood, et al., 2013; Justice, et al., 2008). 

Importantly, it cannot be assumed that Prekindergarten teachers understand the pivotal 
role that high-quality pre-reading instruction can play in the prevention of early reading 
difficulties that place children at risk for RD. Given the weak alignment between school and 
early childhood systems, and inconsistencies in training requirements and supports for 
professional development, Prekindergarten teachers may not know how to strategically 
intervene in ways that help weak skills efficiently “get on track” in time for Kindergarten. To be 
clear, we do not support the view that Prekindergarten classrooms should become “the new 
Kindergarten”. Rather, we recognize the underdeveloped importance of effective and efficient 
Prekindergarten intervention on critically underdeveloped literacy skills that, when left 
unaddressed, negatively impact children’s later reading development. We believe in fully 
supporting Prekindergarten teachers to ensure that children can begin school with solid literacy 
foundations to reduce their RD risk. 

Assessment-Guided Instructional Decision-Making. |ncreasingly, schools engage in 
prevention frameworks that involve the use of assessment data for making instructional 


decisions, as part of an ongoing process to improve learning (Hamilton, et al., 2009). However, 


providing access to reading achievement levels through assessment data is not enough— 
teachers must be clear about how to effectively interpret the results for setting meaningful 
goals and initiating beneficial supports (Stecker, et al., 2008). As an early example of a data- 
based decision-making model in the schools, Response to Intervention (RTI) frameworks pair 
universal screening practices with increasingly complex tiers of intervention for addressing 
children’s different academic learning needs (e.g., Fuchs & Fuchs, 2006;). Adaptations within 
early childhood (e.g., Buysse & Pelsner-Feinberg, 2010; Greenwood et al., 2011) also employ a 
tiered model approach with collaborative problem-solving around assessment results and 
progressively intensive instruction to meet diverse learning needs. More recent prevention 
models have expanded the RTI academic focus to also build positive behaviors needed for 
learning, as well as outline principles for formalizing a school’s cultural system, known as Multi- 
Tiered Systems of Support (MTSS; e.g., Sugai & Horner, 2009). 

A recent position paper released by the Division for Early Childhood (DEC) Council for 
Exceptional Children suggests emerging interest in applying a MTSS approach in 
Prekindergarten (DEC, 2019). Although a discussion of this particular MTSS model is beyond the 
scope of this report, the approach includes an emphasis on (a) what is taught; (b) where 
instruction takes place; (c) when instruction is delivered; and (d) how instruction unfolds (DEC, 
2019). The DEC proposed model, similar to RT| and MTSS models in primary schooling, 
identifies five important process steps for making data-based decisions about strategically 
moving children across three tiers of increasingly intensive support: gather information, 


document, summarize, analyze, and interpret. In addition, it explicates four key practices: 


universal screening, differentiated instructional goals or outcomes, tiered supports, and 
ongoing progress monitoring (DEC, 2019). 

Similarly, but independently, we aimed to develop the LRA Greenhouse to help teachers 
simply and meaningfully use assessment results to identify, address, and support “High Priority 
Need” RD risk factors during the Prekindergarten year for positively impacting children’s 
emergent reading development in Kindergarten. Our primary goal was to help Prekindergarten 
teachers use evidence-informed practices (the “what”) that were developmentally appropriate 
in feasible ways (the “where”, “when”, and “how”) to promote timely learning. To do this, we 
emphasized the development of fundamental pre-reading (rather than exhaustive literacy) skills 
and classroom self-regulation behaviors needed for learning. The co-learning of these two 
competencies is especially important because of their long-term connection in how children 
approach literacy development, and in the process, shape how teachers support their learning 
(McClelland & Cameron, 2012; Pianta & Stuhlman, 2004; Saez, et al., 2012). Children who 
demonstrate greater cognitive and behavioral control (or self-regulation) positively impact their 
skill development directly, and also indirectly through beneficial classroom relationships with 
peers and teachers, which makes their coupled learning an important mechanism for 
supporting school readiness (Blair & Raver, 2015). 

We additionally highlighted children’s learning needs related to cognitive processing 
constraints that hamper literacy development and are associated with RD (e.g., see Johnson et 
al.’s meta-analysis of learning disability cognitive processing deficits; Johnson et al., 2010). The 
identification of potentially interfering cognitive processing weaknesses can enhance teachers’ 


understanding of specific ways to promote efficient learning, which is especially important for 


generating progress. Thus, our goal was not to increase children’s cognitive abilities, but rather, 
address “hidden” factors associated with learning difficulties. We believed that this multi- 
dimensional view of RD risk (and how each dimension contributes to different learning needs) 
was especially warranted during the Kindergarten transition when Prekindergarten instruction 
focuses on growing well-rounded school readiness. In addition to promoting school readiness, 
this approach is aligned with the early childhood field’s desire for strategies that increase 
professional accountability in teachers, and social and emotional development in children 
(Wesley & Buyssee, 2003). 

Finally, we assumed that Prekindergarten teachers could be trained to effectively and 
sustainably implement intentional evidence-informed practices if provided adequate supports 
that fit their needs. Preschool teachers use a wide-range of curricula and instructional practices 
(Pretti-Frontczak, et al., 2002). Therefore, the LRA Greenhouse was designed to flexibly co-exist 
with teacher practice; as such we view it as a supportive and generative tool for simultaneously 
enhancing Prekindergarten literacy skills development for all children and preventing RD for the 
most vulnerable children with identifiable “high priority” needs. We worked closely with 
Prekindergarten teachers to streamline and scaffold their intentional instruction, providing not 
only pre-implementation online training, but also in-the-moment supports. Technology was 
leveraged to simplify assessment administration, assessment results access, interpretation and 
management, instructional delivery, and support for generating learning progress. Below, we 
describe the development, field-testing, and associated data collection of the LRA Greenhouse. 


We discuss the LRA followed by the wrap-around Greenhouse support components. 


LRA Development 


Initial Test & Item Development 
Administration Modality Decision-making 

Addressing the need for teachers to have immediate access to assessment information 
for guiding instruction was a primary LRA development goal. Consequently, test item delivery 
through tablets was an intentional administration strategy for four main reasons. First, the 
tablet medium allows teachers to streamline data collection, ensure its integrity, and provide 
real time data for use in facilitating instructional programming. Teachers have limited time for 
test administration and data-based decision making (Roehrig, Duggar, Moats, Glover, & Mincey 
, 2008). Because the LRA is a self-contained tablet-based tool (using audio guidance and image 
selection), it requires minimal training for test administration (Sdez & Pilger, 2016), making it a 
more flexible option for gathering information about children’s abilities than classroom 
observation methods. 

Second, we also anticipated greater efficiency through the use of computer technology, 
compared to paper-based performance assessments, because multiple children can be 
simultaneously assessed in small groups (with headphones) using the LRA. For example, Carson, 
Gillon, and Boustead (2011) found that a computerized assessment administered at school 
entry was more efficient (taking 20% less time) than a paper-based version administered with 
an examiner. Prior research documents the benefits of touch screen technology use with young 
children to predict early risk for learning and attention difficulties (Berger, et al., 2000; Gaggi, et 


al., 2012; Groot, et al., 2004). 


Third, we anticipated greater feasibility for preschool classrooms with tablet 
administration because these devices are generally affordable ($50 - $100 each) and portable. 
Unlike schools, prohibitive material costs can prevent early childhood centers from using 
beneficial tools, particularly if they lack government funding. Tablets have a variety of uses, 
beyond test administration, that may provide added investment value for early childhood sites. 

Fourth, image-based tablet administration is developmentally appropriate, and a 
pervasive technology in many Prekindergarten homes. Via technology, even complex concepts 
can be reliably assessed easily through simplified language, format repetition, automated 
guiding prompts, and basic response demands. For example, children can respond to all items 
using the touch of their finger (i.e., no mouse or keyboard is required), enabling them to fully 
focus on the mental aspects of each task. 

Measurement Domains Identification 

The development of the LRA was initiated in 2015, based on an increased need within 

the state of Oregon for a statewide Kindergarten assessment to evaluate school entry skills, as 


well as better tools for preventing Kindergarten (and 


Kindergarten Reading 


beyond) learning difficulties. Three primary constructs 
Readiness 


served as the basis for LRA measurement: pre-reading 
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Behavioral 
Regulation 
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skill, classroom behavior regulation levels, and working 
memory (WM) capacity. We focused on these 
constructs because of their unique, yet related, 


Working Memory Capacity 
contributions to RD risk in young children. The figure 


(on the right) depicts these construct relations. 


Pre-reading literacy skills and behavioral regulation are critical contributors to 
Kindergarten reading readiness, both of which are enabled by WM capacity to effectively 
develop skills and regulate behaviors while learning. Weaknesses in emergent reading skills and 
classroom behavioral regulation can both impair reading development by recurrently 
hampering learning, and their relation to each other is reciprocal (Goble et al., 2017). For 
example, first-grade children with RD exhibit greater behavior problems in third grade than 
their peers, and children exhibiting poor task engagement are more likely than their peers to 
experience reading difficulties in third grade (Morgan et al., 2008). 

Coie and Krehbiel (1984) described four ways in which academic and behavior problems 
interact to disable learning: (1) behavior challenges limit instructional access, (2) limited 
attention hinders both behavioral and instructional efforts, (3) early academic challenges 
increase social rejection, and (4) inadequate progress of academic interventions leads to poor 
behavioral responding. The second interaction in the list above aptly characterizes how WM 
deficits undermine behavior and academic learning through its moderation of children’s 
attention as they learn to read (Kofler, et al., 2018; Saez, et al., 2012). For example, weak WM 
capacity can impair children’s learning opportunities by repeatedly limiting the information 
they receive, attend to, and express (Morgan, et al., 2019). Furthermore, WM predicts reading 
achievement and behavior regulation (Nguyen & Duncan, 2019), highlighting its importance as 
another important early indicator of RD risk. Below, we operationalize the LRA constructs (pre- 
reading skill, behavior regulation, and WM) and describe how they are measured. 

It is important to note that the LRA was designed to capture early risk for RD in a multi- 


dimensional, developmentally appropriate manner. We focused on critical precursors (pre- 


reading skills) and learning supportive factors (behavior regulation and WM processing 
capacity) that, when underdeveloped, create hinder reading skill development. Beginning 
Kindergarten with significantly weak pre-reading skills, however, does not solely determine the 
emergence of RD. Rather, it is the combination of significantly weak skills and learning-related 
behaviors (both of which are influenced by WM processing capacity) that suppress efficient 
development and undergird increasingly complex learning difficulties that can become disabling 
(and hard to “catch up” with on-grade learning expectations; Torgesen, 1998). Thus, we 
assumed that providing Prekindergarten teachers with information and training in these three 
areas (emergent pre-reading skill, classroom behavior, and executive WM processing) would 
create separate, but related, directions for making impactful instructional decisions to address 
children’s early learning needs in time for more successful starts to Kindergarten. 

Pre-reading literacy measurement was composed of alphabet knowledge and 
phonological sensitivity tasks, based on the National Early Literacy Panel findings (NELP, 2009) 
and RD research base. Phonological sensitivity refers to the ability to detect and manipulate 
phonemes in spoken words (Anthony, et al., 2003; Treiman & Zukowski, 1996). This sensitivity 
ranges along a continuum, in which listeners become more refined in their processing of 
smaller word parts (e.g., blending or segmenting words > blending or segmenting onset/rime 
word parts > blending or segmenting phonemes; Anthony & Lonigan, 2004; Phillips, et al., 2008; 
Schatschneider, et al., 1999). Greater sensitivity benefits sound-to-print mapping through more 
refined sound detection and linguistic manipulation capabilities (i.e., playing with sounds), an 


ability used in emergent decoding (Lonigan, et al., 2000; Wagner, et al., 1997). 


In addition, effective beginning readers rely on alphabetic knowledge to translate oral 
language to print. Delays in the development of phonological skills can have long-lasting 
impacts on reading development (Boscardin, et al., 2008; de Jong & van der Leij, 2003). Letter 
name knowledge develops first and facilitates the emergence of letter sound knowledge (Kim, 
et al., 2010; McBride-Chang, 1999), providing a necessary alphabetic label for phonetic sounds. 
Letter sound knowledge, however, more strongly predicts beginning word reading 
(Schatschneider, et al., 2004), and weaknesses in Kindergarten create early and lasting 
disruptions to reading development (Boscardin et al., 2008; Saez et al., 2016). Thus, we 
considered letter name and letter sound knowledge separately as measurement targets. 

Behavior regulation measurement was composed of task and prosocial engagement 
subdomains. Task engagement emphasizes factors that impact on-task behavior and task 
completion such as: sustained attention, memory for routines, demonstrations of confusion, 
and independent completion of multiple steps. Prosocial engagement emphasizes factors that 
impact active classroom participation, including: cooperation, sharing, emotional self-control, 
and help-seeking. These subdomains were drawn from research focused on WM processing 
difficulties associated with learning disabilities, Kindergarten readiness, and early childhood 
approaches to learning (e.g., Bierman, et al., 2008; Bronson, et al., 1995; Duncan, et al., 2007; 
Gathercole, et al., 2008; La Paro & Pianta, 2000; McClelland & Morrison, 2003; Rimm-Kaufman, 
Pianta, & Cox, 2000) because of the significant challenge that children face when transforming 
into successful classroom learners for school (Rimm-Kaufman & Pianta, 2000). These two 
aspects of classroom engagement interact, but also play separate roles in how children self- 


regulate and learn. 


Emerging behavior regulation in the classroom (and in particular, the ability to regulate 
task completion and prosocial relations within the classroom) are vital to the quality of a child’s 
classroom experience (Hamre & Pianta, 2001; Williford, et al., 2013). The effective execution of 
“learning-related” (McClelland, et al., 2006) and prosocial behaviors reflects the crucial ability 
to balance one’s emotional and cognitive state required for self-regulation and internal well- 
being (Blair & Diamond, 2008). Effective engagement of goal-directed learning activities 
requires that children increasingly attend to and control their behaviors to adopt valued 


| “ 


classroom routines, school-wide rules, learner “habits of mind”, and social “rules of 
engagement”. Such engagement involves integrated cognitive and emotional circuitry 
management for responding to these demands (Bierman, et al., 2008; Blair & Diamond, 2008). 
Thus, behavioral regulation of both task and prosocial engagement encompasses the control 
and maintenance of arousal states in support of goal attainment, and often they work in 
concert during active learning (when learning has an intentional goal orientation). 

Better behavioral regulation reliably predicts better beginning reading outcomes 
(McClelland, et al., 2000). In addition, emotional regulation also predicts Kindergarten 
achievement (Howse, et al., 2003). Self-control substantially impacts the quality of a child’s 
classroom experience through its shaping of effective interactions during cooperative group 
work, through listening to others and following directions, and by taking turns and sharing, 
which invite future social-learning opportunities. Because the ability to regulate one’s behavior 
is relatively stable (McClelland & Morrison, 2003), it helps to cultivate motivation, peer 


|” 


acceptance, and teacher support. For example, higher “effortful control” predicts better 


Kindergarten reading and school enjoyment (Gaias, Abry, Swanson, & Fabes, 2016), along with 


social competence and likability among peers (Denham, et al., 2003). Positive teacher-child 
relationships are also associated with better beginning reading achievement over time (Pianta, 
et al., 2008). Effective behavioral regulation is associated with better school adjustment 
through its positive effect on the emergence of internalizing (sadness, anxiety, and social 
withdrawal states that can lead to inattention) and externalizing (aggressiveness, impulsivity) 
problems (Eisenberg, et al., 2001). 

Working memory (WM) measurement comprised a typical span design (Conway, et al., 
2005), in which information to be remembered is presented, followed by a distraction (to 
invoke retention + concurrent processing), and then recalled. Our view stems from General 
Capacity Models (e.g., Turner & Engle, 1989), in which the ability to concurrently “work with” 
different mental information is based on individual differences in WM capacity, independent of 
the specific kind of information (e.g., linguistic, emotional, semantic, or visual-spatial) being 
processed (Courtney, 2004). Our use of WM span in the LRA corresponds with Baddeley’s 
conception of the central executive (Baddeley & Hitch, 1974), which is a domain-general 
capability. 

As one of three critical “executive” functions (Miyake, et al., 2000), WM plays a 
particularly important role in learning because it significantly determines processing capacity 
for attention (Engle, 2010; Swanson, et al., 2006). Strong WM results in efficient mental 
processing of multiple pieces of information, effective resistance to interfering thoughts or 
stimuli (or focused attention), and the successful completion of goal-directed complex task 
performance (e.g., information integration, engaging in multi-step activities). Weak WM 


increases one’s susceptibility to mental overload, often manifested as distractibility, 


forgetfulness, confusion, and feeling “overwhelmed”. Low WM capacity is a stronger predictor 
of later achievement outcomes than IQ (Alloway & Alloway, 2010) and is associated with RD 
(deJong, 1998; Gathercole & Pickering, 2000; Siegel & Ryan, 1989; Swanson & Saez, 2003). WM 
is not recruited for the direct retrieval of long-term stored information, but instead, enables the 
“working with” of information that underlies coordinated thinking, and is separate from short- 


|” 


term retention (Bayliss, et al., 2005). This unique “general” role, when weak, can disrupt 
learning in ways that “quietly” impair effective learning (e.g., by impeding mental processing 
needed for complex task activities and self-regulation). 

By Prekindergarten, the three core executive functions (WM, inhibitory control, and 
mental shifting/flexibility; Miyake et al., 2000) emerge as key players in how children learn, 
including by regulating attention (Garon, Bryson, & Smith, 2008). WM plays a critical role in 
attention regulation by keeping necessary information current and “in mind” despite 
distractions or interference (Conway, et al., 2005). Through children’s emerging abilities to 
simultaneously process different information, they gain greater control over their internal 
experiences (i.e., feeling and thought attentional regulation) and behaviorally regulated 
responses, benefitting their learning within a school environment (Blair & Diamond, 2008). 

WM mediates the relation between classroom inattention and poor reading (Rogers, 
Hwang, Toplak, Weiss, & Tannock, 2011). Inattention is a stable predictor of later poor reading 
achievement (Duncan, et al., 2007), and has been found to interfere with phonological 
processing skills needed for reading development through its role in Kindergarten children’s 
failure to adequately attend to the phonemic structure of language, resulting in protracted 


poor word recognition (Dally, 2006). However, low WM is also associated with teacher ratings 
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of behavior dysregulation, including student inattention, high distractibility, problems with 
monitoring work quality, and difficulties with problem-solving (Gathercole, et al., 2008). 
Relatedly, and crucial to LRA development, Preschool children who have difficulty managing 
their “learning-related” behaviors are often considered less “ready to learn” in Kindergarten by 
teachers (Lin, et al., 2003), and students with weak WM in Kindergarten continue to experience 
these difficulties into elementary school (Alloway, et al., 2009). The role of WM and attentional, 
then, on Kindergarten reading is both general (or “executive”), and skill specific (Welsh et al, 
2010). 

Individuals with weak WM capacity are also less able to suppress their emotions 
(Schmeichel, et al., 2008), which makes behavior regulation challenging, particularly when 
emotional arousal is high. There are, however, environmental factors, such as noise level, that 
can also subtly increase children’s cognitive load and weaken their ability to effectively function 
in the classroom. Even among children as young as three, how well background noise is filtered 
out predicts later reading outcomes (White-Schwoch et al., 2015). In other words, loud 
classrooms contribute to cognitive “noise” that children must inhibit in order to focus on 
learning. For children with weak WM capacities, the effort of focusing attention to learn while 
also inhibiting competing distractions makes them more likely to become mentally “flooded” or 
overwhelmed because their “brains become too full” sooner. Therefore, in our view, WM 
processing plays a critical role in both new literacy skill development (through the concurrent 
processing needed to make conceptual and linguistic connections in learning) and behavioral 
regulation needed for effective classroom learning. We note that WM processing span is used, 


rather than other WM indices, because of its interpretation ease in relation to implementing 
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environmental “thinking support” strategies to aid Prekindergarten learning (Conway, et al., 
2005). 
Item Generation 

All Pre-reading items were formatted for audio-visual multiple-choice delivery, using 
three picture-based response options shown with audio prompting. For the phonological 
sensitivity items, each image is announced when it is shown on the screen to ensure children’s 
correct recognition of all response options. A practice item with bifurcated response feedback is 
provided to clarify task demands (i.e., children receive oral and visual feedback on their 
response, to either validate a correct response or gently reinforce the correct answer for an 
incorrect response). All items are scored 0 (incorrect) or 1 (correct), and summed to compute a 
total score. 

We based the range and depth of Pre-reading items on the Common Core State 
Standards in Kindergarten (later refined using the Oregon State Early Learning Guidelines; 
Oregon Department of Education, 2016), and published reports of item difficulty (e.g., Drouin, 
et al., 2012; Justice, et al., 2006; Phillips, et al., 2012). A Kindergarten teacher and elementary 
reading specialist were hired and trained to assist with item writing, as early reading content 
specialists. Ninety-six literacy items (48 alphabet knowledge and 48 phonological sensitivity) 
were initially developed in the item bank, which were reduced to 24 working items (16 
alphabet knowledge and 8 phonological sensitivity). Working items were selected based on 
their capacity for imaging, perceived difficulty (based on easy, medium, and hard 


categorizations), and “fit” with the multiple-choice format (enabling one correct answer and 
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two plausible distractors). Each distractor was developed to be either phonologically or visually 


similar to the correct choice. See Pre-reading item examples below. 
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We drew the initial list of behaviors for the Behavior Rating Scale (BRS) from research 
literature (e.g., Bronson, et al., 1995; McClelland & Morrison, 2003), which was reviewed by a 
Kindergarten teacher for wording, developmental appropriateness, and clarity. The revised list 
was then evaluated in five preschool classrooms (n = 75) by two researchers to examine the 
extent to which listed behaviors could be easily observed in Preschool settings. Classrooms 
varied widely in educational philosophy, ranging from daycare (without clear instructional 
objectives) to pre-academic (with clear daily instructional goals), with play-based classroom 
structures falling in between. Two systematic observations, within one month, were conducted 
at each site to examine whether behaviors were observable across: (a) diverse sites, (b) 
genders, and (c) common routines (e.g., Circle, Snack, Project, Transition, Story). In addition, 
each behavior was evaluated on the extent to which a teacher might feasibly and reasonably 
notice variation among children given the frequency of occurrence, bustling environment, and 
differing task elements. Teacher delivery of directions for these activities was also examined 
across sites to clarify potential variation and constraints on children’s opportunities to learn 
pre-academic skills and prosocial behaviors in the classroom (e.g., the extent to which multi- 
step directions are used, thereby shaping task engagement requirements, across settings). 

A five-point scale (1 = definitely not; 5 = definitely yes) was used for the behavior ratings, 
including an option for “no basis for rating” (0), with the generic prompt, “Does the child...”. 
Items were presented by a displayed target behavior (e.g., “...need additional reminders of 
what to do?”), in which teachers rated their entire class before moving to the next behavior 
prompt. This whole-class design was used to help minimize responder drift (by reducing the 


potential for “halo” or “horn” effects for specific children) and to increase rating completion 
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efficiency by keeping teachers’ attention focused on the same behavior for the entire class at a 
time. The BRS contained 30 items (15 task engaged and 15 prosocial behaviors), with a total 


possible score of 150 points, 75 points per subdomain. See a BRS item example below. 


Behavior Rating Scale 


Does your student... 


DEFINITELY DEFINITELY 
No Basis for Rating ie es seal 


Brown, Andres 


a 
(=a 
For each student, ix 
(i 2 


1. Touch student's name 
2 Touch a rating number {1-6 or NER} 
‘When ratings for all students on your fet have 
bean completed for a behavior, touch 
‘Save & Continue” to provide ratings for the 
next behavior. 


The Working Memory (WM) task was developed using a span design for two main 
reasons: (1) span accounts for performance even when processing efficiency is controlled 
(Gavens & Barrouillet, 2004), and (2) the theorized, positive association between span and 
information quantity that can be held concurrently in mind directly applies to classroom 
functioning (making it more useful in practice for assessment-guided instructional decision- 
making). Thus, a larger obtained span confers a greater capacity for mental processing, even in 
children as young as four years old (Gathercole, Pickering, Ambridge, & Wearing, 2004). 
Conversely, a smaller obtained span reflects a more limited capacity for mental processing, and 
a corresponding need for teachers to provide “thinking supports” to facilitate strong learning. 

Using a definition of “holding information in mind while processing something else”, the 
WM task was designed with the following three steps for gradually increasing set sizes: (1) 


children are shown a stimulus (“Buddy”, a juggling monkey who drops 2 to 5 objects, depending 
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on the span set size, up to 5 levels), (2) a distractor (Buddy doing something novel on the 
screen, like driving a car), and then (3) prompted to recall (the selection of Buddy’s dropped 
items in the order that he dropped them). Examinees are provided with three practice items 
with bifurcated response feedback to clarify task demands before proceeding to the actual test. 
In this way, practice items teach children to attend to and remember (a) what Buddy is juggling 
and the object dropped, (b) multiple objects that may be dropped, and (c) the order in which 
objects were dropped. WM test items are scored 0/1 with total span scores ranging from O 
(unable to accurately process 2 items concurrently with distraction) to 5 (able to process 5 
items concurrently with distraction). To minimize frustration, a stop rule was used to cease 


testing when 2 of 3 items are wrong within a level. See a WM practice item example, below. 


Field Testing & Item Refinement 
Internal beta-testing was undertaken for each phase of LRA test and item development. 
The initial prototype was developed in black and white, and compared with black and white 


paper and pencil administration with n = 50 Kindergarten students to examine: 


(a) the extent to which tablet technology was feasible for use among younger children, 
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(b) the efficiency of paper-pencil compared to audio-guided tablet administration, and 


(c) the motivational impact of black and white presentation given the age of examinees. 


For an initial field-test, children were drawn from four classrooms located in Southern Oregon, 
children in two AM and two PM half-day classrooms were counterbalanced and assigned to 
either individual paper-pencil or tablet administration. 

Children rated their interest in completing items before and after administration, using 
a 1 (strongly dislike) to 5 (strongly like) point scale. In general, children rated their experience 
similarly (4/5) pre- and post-administration. Item analyses (of time and performance) suggested 
specific areas for refinement, and observations of student performance indicated that children 
were able to complete the tablet version with minimal teacher support and that the bifurcated 
practice item feedback functioned as intended. Children interviews revealed an expressed 
desire for the assessment to more closely approximate personal experiences with tablets and 
smartphone games by using colored images, which explained the similar ratings found across 
administration type (i.e., personalized human administration was rated as positively as black 
and white tablet technology use). 

A revised version of the LRA Pre-reading items was subsequently developed using 
colored images, and field-tested in two Kindergarten classrooms (one AM and one PM, with n = 
42 students) in small groups of 4-6 children using headphones. Children, on average, completed 
the 60-item (including 6 practice items with feedback) test in 13.70 minutes. From this work, 
important implementation needs, such as the use of visual barriers, smaller headphone sets, 
and the role of prior experience for “touching” the correct answer were identified (e.g., 


tentative/hesitant, swiping, or touch users were observed to differentially respond to audio 
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commands). Additional item analyses (of time and performance) indicated areas for test item 
refinement. In addition, we solicited feedback from three Kindergarten teachers on test items 
and screen elements (considering functionality, design, and user-friendliness). 

After item refinement, a third field-test was conducted with n = 15 Prekindergarten 
children (in small groups with “kid-sized” headphones and visual dividers) using a shortened 
test form that included representative “easy” items from the Kindergarten battery. Our goal 
was to examine: (a) whether tablet administration was similarly feasible for Prekindergarten 
children and (b) the extent to which anchor pre-academic and behaviors items could be shared 
between the two age groups. On average, Prekindergarten children responded to items within 
4.6 seconds, one second slower than Kindergarten participants (when the same “easy” items 
were compared). In general, Prekindergarten children completed the assessment similarly to 
Kindergarten students in small groups; however, touch submission reinforcement, task focus 
redirection, and reassurance during uncertainty were more frequently observed, as expected 
for younger and less experienced Prekindergarten children (only about 25% reported previous 
tablet use). Administration observations indicated that children similarly understood and 
positively responded to practice feedback (Saez & Pilger, 2016). 

In addition, three Prekindergarten teachers evaluated the BRS items. Teachers were 
asked to indicate which children were perceived to be “high, average, and low” achieving within 
their class, which was cross-checked against their initial behavior ratings. Teacher feedback was 
used to refine item wording to better fit early childhood classroom activities. As a result, five 
behaviors were revised, unclear language was refined, and modifying words were moved to the 


beginning of all prompts (e.g., “Gets distracted easily” was changed to “Easily gets distracted”). 
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Initial field-testing of the WM task included cognitive labs conducted with n = 4 
elementary school children, which indicated that the original test item design using same- 
colored abstract and conventional shapes would be too difficult for Prekindergarten children 
(e.g., cognitive lab interviews revealed a “common object renaming” strategy used for recall). 
Consequently, all non-practice test items were transformed into common objects (e.g., the 
triangle was changed to a strawberry) that were thematically related (e.g., fruit) to enhance the 
task’s developmental appropriateness. The revised version was field-tested with n = 3 
Kindergarten and n = 3 Prekindergarten children. Practice and test items functioned as intended 
and we found a typical range of span performance (0-3) for this age group. 

Scale Development 

Teachers may choose to administer either the Pre-reading or WM tasks first, and item 
delivery can be stopped and resumed at any point. To begin, teachers select a child’s name 
from the pre-registered class list and then the task. All child-administered tablet tasks begin 
with a simple multiple-choice image matching item (with bifurcated feedback) to familiarize 
children with the assessment format. When children complete a task, an image and audio file 
let them know that they are “all done!” Intermittent “keep going” audio prompts are provided 
at 30-second intervals for items in which no response has been submitted. When one task is 
completed, teachers unlock the screen to complete the two-step process again for the 
subsequent task. Teachers complete the BRS within the same testing window. They may enter 
and exit at any point after a given behavior has been completed for the class (i.e., ratings for all 


class members must be saved prior to exiting the BRS). Upon return, teachers begin rating after 
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the last completed behavior. Teachers may complete this measure using a tablet or computer 
(through the support website, described below in the LRA Greenhouse section of this report). 
Preliminary Risk Cut-Score Determination 

The final version of the LRA battery was field-tested in a Prekindergarten class with n = 
15 children to identify an initial RD risk cut-score. This class was again administered the battery 
at the end of the school year. On both occasions, children were assessed in small groups within 
a one-week period, and the BRS was completed by the teacher within that same time frame. 
Children engaged in initial drafts of Greenhouse activities (described in the LRA Greenhouse 
section below) between the two assessment time points. An in-depth teacher interview was 
conducted to ascertain the teacher’s perception of score meaning. For example, the teacher 
was asked about her understanding of specific scores in relation to others, how she could use 
the information to support enhanced learning, and likely un-assessed factors contributing to 
score results. In addition, the teacher ability-grouped her class for each domain and provided 
extended descriptions for characterizing different performance levels within her class. 
Descriptive item analyses were conducted using combined data additionally drawn from n = 33 
Prekindergarten children during prior field-testing, to inspect performance variation on final 
items within the literacy measure. Time; and Time; differences were compared, and teacher- 
derived group differences were examined to create an initial “risk” cut-score. 

The initial Pre-reading cut-score was determined based on preliminary analyses of the 
score distribution for n = 30 children across fall, winter, and spring time points. A broad “risk 
range” was targeted for performance between the 20"" and 40" percentiles, consistent with the 


RD risk literature (e.g., Simmons, et al., 2008). We used 10 as the cut-point score across all 
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three time points to gradually narrow risk identification to between the 20" and 30" 
percentiles by the end of the year, and provide teachers with a consistent and clear benchmark 
for interpreting performance (with the goal to subsequently conduct IRT analyses with larger 
samples). Appropriately, follow-up examinations indicated that the risk cut-point fell between 
the 40" and 50" percentiles in fall, at the 30" percentile in the winter, and between the 20" 
and 25" percentiles in the spring. Children scoring below the Pre-reading cut-point are 
identified as at-risk on the “High Priority Needs” Report by name in black. For at-risk children 
whose WM performance is also zero (indicating that at the first level, at least two items cannot 
be held in mind despite distraction), their name is shown in red (instead of black) to indicate 
higher priority literacy-learning support needs (i.e., pre-reading skill + WM processing). 
Similarly, for the BRS, we targeted the 25" percentile as a stable cut-point (total score of 
93 in the fall) for low behavior regulation that increases children’s risk for disrupted reading 
development. Total scores (rather than subscale scores) were used; however, the cut-point 
later proved unsatisfactory during piloting and is currently not used to identify RD risk. As a 
final check on the appropriateness of “RD risk” classification, we examined face validity (i.e., 
score meaningfulness based on teacher perception of classroom experience), and asked four 
teachers to rank-order children in their classes into ability groups for each of the measured LRA 
constructs and provide defining characteristics for each group. Teacher RD risk grouping, on 


average, agreed with the LRA risk identification 82.2% of the time (Hinkle, et al., 2018). 
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Internal Consistency 

Cronbach’s alpha was computed to examine internal consistency for the LRA Pre- 
reading and BRS measures. Because of the WM span design, all items were not administered to 
all children (i.e., testing is stopped with two errors within a level), and thus, alpha could not be 
computed. For the 26-item Pre-reading measure, alpha was .79 in fall, .73 in winter, and .76 in 
spring. For the 30-item BRS, alpha was .97 in the fall, .94 in the winter, and .95 in the spring 
(see Appendix, Table 3). 

Cross-Measure Relations 

Within-measure Pearson correlation associations were examined across fall, winter, and 
spring administrations. We summarize these findings in the detailed correlation matrix shown 
in Table 4 of the Appendix. Pre-reading inter-correlations ranged from .73 - .80 across seasonal 
administrations. Within the BRS, task engaged item correlations ranged from .74 - .81 and 
prosocial engaged item correlations ranged from .81 - .89. WM span score inter-correlations 
were weaker, ranging from .17 - .28, a challenge previously reported by other researchers 
(Cowan, 2010; Gathercole, et al., 2004). 

Relations between measures were also examined. Pre-reading correlated r= .18 - .56 
with WM span scores, r= .38 - .69 with task engaged behavior ratings, and r= .36 - .59 with 
prosocial engaged behavior ratings (r = .40 - .71 with the BRS total score). WM span scores 
correlated r= .05 - .53 with the BRS total score. Task engaged behavior ratings correlated r = 
.12 - .49 with WM span scores and r= .52 - .77 with prosocial engaged behavior ratings. 
Prosocial engaged behavior ratings correlated r= -.05 - .57 with WM span scores. Winter WM 


scores yielded unexpectedly low associations with the other measures. 
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Teacher Report 

For all implementing classrooms, we conducted systematic interviews to gather 
information about Prekindergarten teachers’ perceptions of the LRA. In general, teachers 
reported positive perceptions of their experience using the LRA regarding test completion (child 
tasks and teacher behavior ratings), content, and score use. See Tables 5 - 9 in the Appendix for 
survey responses concerning various aspects of the LRA Greenhouse user experience (including 
specifically about the assessment component). In general, teachers positively rated the LRA’s 
efficiency, ease of navigation, and usefulness of results. They unanimously agreed that 
measured pre-reading skills, classroom behaviors, and WM span were developmentally 
appropriate. Below, we provide representative quotes that capture the consistent themes 
found. 


“The assessment was quick and the information got back to you quickly. | liked being able to 
assess them on the tablet- so | wasn’t assessing them. | didn’t have to score it and [could just] 
review it.” 


“| used them [LRA scores] as a weekly reminder to see those ones that are struggling. If you’re 
not focused on them, you can forget, so the scores helped me see who | needed to focus on, 
who needed extra support- it helped me be pro-active.” 


“It’s good to know that they’re getting it and the ones who didn’t [over time], so you can keep 
working on it. Because it fassessment] was more often in the year, | could keep tabs on how 
they’re doing.” 


“The curriculum had it ready to implement and | had a real [assessment] score for me to 
understand, and then there was 3 terms to work with, and | could see the progress.” 


“When you're a teacher, and in constant motion, it's hard to see the overall big picture of each 
child's development. [The reporting] gave me a Starting place at the beginning, and middle of 
the year.” 
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Greenhouse Development 


In 2016, we were awarded funding from the Office of Special Education Programs 
(OSEP) for a project entitled “Project ICEBERG” (#H327S150007). The purpose of Project 
ICEBERG was to identify implementation strategies for sustainably supporting evidence- 
informed practices across the Kindergarten transition using the LRA. Because the LRA was 
designed to facilitate Prekindergarten data-based decision-making, we aimed to build 
technology- and classroom-based strategies for empowering teachers to make effective 
“assessment-guided instructional decisions” for reducing children’s risk for RD (Buysse & 
Pelsner-Feinberg, 2010; Greenwood et al., 2011), as well as generally strengthen children’s 
literacy skills in time for Kindergarten. In addition, we strived to strengthen systems-level field 
supports for capable implementation of data-based decision-making practices within early 
childhood (Ratsavong, et al., 2017). Our primary goals included refining the LRA and developing 
reports to facilitate Preschool teacher reflection and action about children’s “High Priority 
Need” risk factors, as well as identify and incorporate strategies and supports to sustainably 
and feasibly guide instructional decision making without unwieldly investments in training. 

Therefore, our work was not focused on developing a “new” intervention; instead, the 
focus was on helping teachers to effectively implement practices known to work (Diamond, 
Justice, Siegler, & Snyder, 2013). This included developing and refining: an app for housing the 
assessment, reports, and instructional supports to enable efficient and effective teacher action; 
a support website to provide computer access for the tablet app, house online training 


modules, provide curriculum access, and store guidance products to scaffold feasible and 
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sustainable implementation; and all necessary content to enable effective assessment-guided 
instructional decision-making. 
Theory of Change Model 

Our theory of change model (see figure, below) reflects four critical teacher actions and 
three intended outcomes to help achieve the goal of successfully implementing the LRA in 
Prekindergarten classrooms. First, teachers participate in online training for educating them 
about their roles as decision-makers to prevent RD and key assessment-for- instruction 
decision-making concepts. Second, teachers administer the LRA at designated seasonal time 
points to identify early RD risk (“High Priority Needs”) and provide intentional instruction as 
children engage in curriculum activities. Third, teachers implement Greenhouse activities with 
all children and individualize prompting, support, and monitoring for children identified with 
High Priority Needs. Fourth, teachers evaluate children’s progress toward successful reading 
development outcomes and reduced RD risk and access additional resources to extend their 


understanding and use of targeted practices. 


Intended LRA Greenhouse System Components Intended 
Population Outcomes 
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This theory of change model led to the development of a 5-step assessment-guided 
instructional decision-making model: Assess (emergent pre-reading, behavior regulation, and 
working memory levels), Interpret (reported performance), /nstruct (using intentional evidence- 
informed strategies for all), Monitor & Evaluate (the progress of children, particularly those 
identified with High Priority Needs), and /ndividualize (provide even greater support, as needed, 
based on the prior two steps to keep progress moving in the right direction). Assessment is 
ideally conducted in the late fall (after teachers have established their typical classroom 
routines), winter, and spring, during program-established two-week time frames, with 
curriculum activities implemented nine days per month between the fall and spring 
assessments (i.e., across six months). This model was originally based on data-based decision- 
making frameworks found in the research literature (e.g., Chard et al., 2008; Fuchs & Fuchs, 
2006; Schatschneider, et al., 2004) and was later informed by survey, focus group, and 
interview work conducted across 10 early childhood sites during field-testing. 

Strategic Supports Identification 

Feedback from the Field. The types of supports most needed to scaffold effective 
assessment-guided instructional decision-making in Prekindergarten was examined through 
both field surveys and interviews (see Technical Report 1602 for “Innovation Needs Survey” 
findings; Irvin, et al., 2016). This was a crucial, initial step for exploring the feasibility and “fit” of 
the LRA Greenhouse for Prekindergarten use, to help move it from concept to widespread, real 
use (Smith et al, 2014). To summarize this report, n = 50 respondents (a convenience, 


|” 


“snowball” sample of Preschool and Kindergarten teachers and administrators from across the 
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state of Oregon) indicated that the following would be most needed to prevent RD using an 
innovative technological tool: 
(1) Quick and easy access to information for teachers, 
(2) Instructional strategies that were clearly linked to assessment results, 
(3) Support for intentional practices that increase teacher knowledge flexibly and 
enhance planning, and 


(4) Technological means to track ongoing progress for children at-risk. 


Although 80% of survey respondents indicated that early screening for learning 
difficulties is important, they also noted existing screening practices, high-quality curriculum, 
parental action, and funding as obstacles to successful RD prevention. Fewer than 30% of 
respondents agreed that Preschool teachers are currently skilled in using assessment results to 
support children’s learning needs. Given these findings, we revised our initial plans to present 
evidence-informed practices through one-page resources and guidance documents to instead 
create a more scaffolded system. For example, we adapted and systematically organized 
evidence-informed and promising practices within a generative curriculum (activities that 
simultaneously outlined for teachers what to do and inspired them to add to their typical 
practices on their “own days”) rather than provide practice descriptions and overviews. 

The resulting curriculum became the anchor for linking training, assessment use, 
instruction, monitoring, and “need to know” understanding components for effectively 
engaging in the 5-step assessment-guided instructional decision-making model. In this way, LRA 
Greenhouse implementation could “fit” with existing practices and better meet needs within 


the field (i.e., rather than seek to replace current literacy instruction, we aimed to “add on” to 
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existing approaches). This approach was driven by literature documenting the importance of 


addressing teacher beliefs and connecting, rather than opposing, new and used practices to 


enable professional mastery when initiating changes in practice and adopting new innovations 


(Al Otaiba, Hosp, et al., 2008; Ertmer & Ottenbreit-Leftwich, 2010). 


Feedback from Preschool Teachers. We also administered a “Teacher Readiness survey” 


to n= 11 teacher participants prior to their use of the LRA Greenhouse to examine their 


perceptions concerning potential implementation strengths, needs, and barriers that might be 


encountered. Below, are the main themes drawn from the survey results. 


Current Implementation Assets: Existing teacher familiarity with the practice of 
developmental screening and the use of assessment results; well-qualified teachers 
that are already knowledgeable about literacy development; willingness to devote 
20 minutes (per child) for group screening assessment three times a year; site access 
availability to reliable high-speed wireless internet; and interest in having “quick 
access to information” about children’s learning needs to share with both colleagues 
and parents. 

Potential Implementation Needs: Resource availability (e.g., access, funding) to 
implement a high-quality curriculum; and ongoing coaching and technological 
support. 

Potential Implementation Barriers: Staffing changes; finding time/space for 
administering an assessment; knowing how to use assessment information to help 
support children in the classroom; varied teacher experience using tablet 


technology; uncertain administrator support; unclear standards and systems-level 
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support to guide how implementation efforts will back Kindergarten readiness 
efforts. 
Consequently, we focused development on addressing three primary capacity-building needs. 

(1) Curriculum: Create a highly scaffolded flexible curriculum (containing high-impact 
evidence-informed strategies) with support resources to facilitate feasible and 
sustainable implementation, with assessment use for instruction. 

(2) Knowledge & Strategy Access: Enhance teacher knowledge about assessment, 
intentional instruction, and their link to reflective decision-making for driving 
intentional practices for preventing RD. 

(3) Technological Management: Build quick and easy access to technology-enabled 
resources and supportive features to enable “on the job” deeper learning and 
successful 5-step model engagement. 

Although we had previously established that Prekindergarten teachers could administer 
the LRA with minimal training, we learned that the role of guided instruction for teachers would 
be pivotal to effective tool use given the absence of unclear early learning standards and 
Kindergarten readiness expectations. The curriculum drove development to enable an 
important change to teacher practice: ongoing, feasible, and effective action for intentionally 
teaching foundational literacy skills to the entire class while responsively addressing identified 
weaknesses for children identified with “High Priority Needs”. 

In addition, technology was carefully incorporated to enable easy access and reduce 
cognitive demands associated with new learning and tool incorporation over time. 


Prekindergarten classrooms are bustling environments, and teachers have minimal time for 
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activity preparation and reflection, so highly-scaffolded supports were used to score and simply 
report assessment results (to facilitate meaningful interpretation), plan and deliver intentional 
instruction (to facilitate systematic and strategic practices consistently over time), evaluate 
learning while it happens (to facilitate cohesive thinking and action), and access information as 
needed (to bridge gaps in knowledge that may pose barriers to successful implementation). 

Through the exploration process, we identified key practices and tool features that met 
the field’s and teachers’ expressed needs, matched evidence-informed practices to fill data- 
based decision-making gaps in Prekindergarten, identified (and instituted solutions to) barriers 
in policy that might hinder tool adoption, and organized Greenhouse development around 
building implementation capacity and sustainability (Blase, van Dyke, & Fixsen, 2013). We 
evaluated implementation readiness prior to development, as well as during development, 
using systematic interview, focus group, and survey protocols to inform our decision-making in 
building this new system. Below we discuss the development of the resulting instructional 
materials, 5-step model implementation resources, and wrap-around teacher supports, and 
findings from two pilot studies conducted for evaluating LRA Greenhouse use. We engaged in 
over 400 hours of classroom observations of curriculum activities implementation, followed by 
systematic teacher interviews to identify specific and strategic supports (described below). 
Curriculum and Instructional Resources Development 

The curriculum includes systematic classroom-ready activities and printable materials 
comprising intentional learning opportunities for three common classroom routines: circle time 
(EXPLAIN), learning centers (ENGAGE), and group storybook reading (EXPLORE). These activity 


types were chosen because of their generalizability across different programs and mix of group- 
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and individual- opportunities to learn. We assumed that ENGAGE center and EXPLORE story- 
reading could be flexibly implemented to reinforce EXPLAIN learning. Because of stark 
differences in how these common activities are named across the field (e.g., whether “Circle” is 
called “carpet time”, “daily gathering”, “morning meeting”, etc.), we created “new” names to 
foster a common language and help focus teachers’ attention to the function of each activity 
type. Developed activities include both child- and adult-led learning experiences, to provide a 
blended approach that values meaningful engagement of both children and teachers. Activity 
plans are presented within a tablet app, accessed via a monthly planning calendar (explained 
further below). 

Each activity plan includes eight sequentially ordered app screens, although teachers 
can move through them in a different order using the icon-based navigation bar. Where 
appropriate, suggested phrasing and access to the Activity Checklist (for monitoring learning 
progress) is available through pop-up screens. Screen layout between units is identical, except 
in color, text (content), and icons. Although we recognized that teaching literacy skills and 
behavior regulation differs, we organized instructional strategies and content to similarly fit 
within EXPLAIN, ENGAGE, and EXPLORE classroom routines to facilitate efficient teacher 
learning, and endeavored to include both explicit and implicit strategies across both unit types 
to maintain a consistent framework. 

In general, we applied well-known principles of learning science across activities (Deans 
for Impact, 2019), to provide regularly implemented intentional and choice routines; initiate 
opportunities for children to reflect on themselves and their changing worlds; nurture 


expression and promote warm interactions; incorporate playful challenge and increasing 
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complexity; allow for accurate, specific, and efficient teacher feedback while learning; infuse 
teacher modeling and diverse methods of practice for effective skill-building; and foster support 
for children’s co-regulation of learning experiences. Essentially, we aimed to create a flexible 
(i.e., “drop-in”) structure for fostering children’s emergent goal orientations as “students” 
(Bergsmann, Luftenegger, Jostl, Schober, & Spiel, 2013), who effectively interact with teachers 
to learn and not just be “taken care of”. Cultivating this critical relationship is associated with 
positive reading growth from Prekindergarten through elementary grades (Pianta, et al., 2008). 

Initial activity drafts were designed for either a “prompted” or “instructional” 
implementation, with “prompted” activities emphasizing less structured teaching and more 
open-ended engagement (e.g., using child exploration to find the monthly letter using 
environmental print instead of teacher-led explicit instruction with picture cards). Both 
implementation approaches were first studied over the course of two months in a 
Prekindergarten classroom using the same alphabet letter to compare differences in observed 
learning, child engagement, and effective teacher implementation. The teacher was 
interviewed daily after observed implementation to gather feedback about activity instructional 
perceived assets, feasibility (including preparation), and challenges. Based on our findings, we 
selected the best activities from each approach to create a “balanced” activity plan (Connor, et 
al., 2006). We then field-tested the blended approach using a different alphabet letter, and 
once the format was established, began similarly developing behavior regulation activities. 
Instructional Activities 

For each Activity Plan day, across both Literacy and Behavior curriculum units, teachers 


can access five activities: whole group Circle EXPLAIN (with Activity Checklist monitoring for 
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Literacy), whole group Story EXPLORE (with suggested, accompanying book titles aligned with 
the daily learning goal), and three small group Center ENGAGE activities (MAKE, DO, and PLAY). 
Teachers are walked through a 5-step Circle activity via the tablet app, with suggested phrasing 
provided in each step to help strengthen implementation fidelity and sustain intentional 
instruction. For example, teachers introduce the learning goal, engage in group sharing, 
conduct a check in understanding, and end with individual responding for evaluating learning 
and clarifying misconceptions, well-known features of explicit instruction (Archer & Hughes, 
2011). EXPLAIN instruction was primarily designed using explicit, systematic principles because 
they are efficient and reduce cognitive demands for learning (Simmons, et al., 2007; Smith, 
Saez, & Doabler, 2016). Within this basic organizing framework, teachers also demonstrate 
skills, provide examples and non-examples, and keep instruction focused and highly engaging 
(and when suggested phrasing is used, clear and concise; Archer & Hughes, 2011). Personalized 
feedback, monitoring, and scaffolded prompting is encouraged to promote high levels of 
learning success (Archer & Hughes, 2011). 

MAKE and DO activities are table activities designed to provide flexible implementation 
options while still reinforcing daily learning goals. MAKE activities are complex and multi-step, 
involving high teacher facilitation, whereas DO activities are simple and entail significantly less 
teacher engagement (i.e., semi- or fully-led by children). PLAY activities involve either child-led 
dramatic play (for Literacy units) or imaginative, picture-card storytelling (for Behavior units), 
and teachers are trained to “drop in and out” of activities as appropriate. PLAY activities invite 
social, object, language, and pretend skills, which benefit social, emotional, behavioral, and 


cognitive development (Connolly & Doyle, 1984; Halberstadt, Denham, & Dunsmore, 2001; 
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Schwebel, Rosen, & Singer, 1999). Each ENGAGE activity includes a 1-page “table top” teacher 
support to guide preparation and implementation, and provide suggested prompts and 
questions to facilitate authentic intentional talk between teachers and children. Simultaneous 
activity within the classroom can create cognitive overload for teachers (Feldon, 2007); we 
created these 1-pagers to reduce the cognitive demands inherently part of implementing novel 
practices, particularly across differently skilled teachers working in the same room. Each Table 
Topper enables quick understanding about the activity’s purpose, necessary materials, step-by- 
step instructions, and to provide supportive scaffolding questions, target words, or phrasing. 
Teachers are trained on the importance of each type of activity and can access guidance “Get 
Ready” resources on the LRA Greenhouse website to help organize when and how they will 
incorporate LRA Greenhouse activities into their daily routines. Below, see examples of 
curriculum Activity Plan, activity printables, and resources for Literacy (on left) and Behavior 


Regulation (on right) units. 


Literacy Units. Literacy Activity Plans repeat each month for a different letter, with the 
instructional focus shifting across five days—teachers devote Days 1 and 2 to building 
phonological sensitivity, Days 3 and 4 to mapping those sounds to print for building letter sound 


knowledge, and Day 5 to making meaning (creating, comprehending, and using higher-level 
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literacy skills linked to the monthly letter). Activities are implemented across two weeks of 
alternating days, based on early field-testing that compared a consecutive, versus alternating, 
days model. Teacher feedback from the field-test suggested that consecutive days focused on 
literacy learning was less sustainable than alternating days, which is consistent with 
psychological research on the spacing effect (e.g., Dempster, 1988). Subsequent input from 
teachers revealed that they valued having their “own day” in between LRA Greenhouse Literacy 
unit days to “add in” their activities related to the beginning letter focus (e.g., incorporating 
“spring” activities in between LRA Greenhouse Days 1-5 “letter S” activities). 

Each month, activities are repeated to promote teachers’ on-the-job learning and help 
classrooms establish regular literacy learning routines. Specific activities were drawn from the 
What Works Clearinghouse and influential early childhood centers, such as Center for Early 
Literacy Learning (CELL); Center on the Social and Emotional Foundation for Early Learning 
(CSFEL); Promising Practices Network (PPN); and Center on Enhancing Early Learning Outcomes 
(CEELO). We also benefitted from numerous early childhood teacher websites for thematic 
activity and storybook inspiration, as well as for gauging developmental appropriateness. 

We referenced the Embedded-Explicit Emergent Literacy Intervention (Justice & 
Kaderavek, 2004; Kaderavek & Justice, 2004) in constructing a balanced instructional approach. 
This model emphasizes unique contributions of embedded (naturalistic, whole-language) and 
explicit (systematic, skill-focused) activities. Based on the early reading and literacy literature, 
we focused on two skill-building emphases: transferring phonological awareness into beginning 
letter sound knowledge and developing critical thinking skills to support comprehension. These 


skill-building emphases entailed strategically building teachers’ capacity to: 
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(a) Implement explicit activities for bolstering emergent literacy skill development, 

(b) Engage in strategic prompting to sustain learning goals with fidelity over time, and 

(c) Use assessment for guiding their instructional practice. 

The Embedded-Explicit model pre-supposes a multi-tiered instructional paradigm, in 
which children exhibiting struggle during whole-class activities are provided a more intensive 
small-group intervention. In general, the LRA Greenhouse corresponds with this approach, with 
one minor modification: Small-group Center activities (for providing additional learning 
experiences) are available to all children and teachers individualize enhanced supports based 
on observed child needs. For example, for some children, teachers might provide high levels of 
intentional language use during PLAY, whereas for others, they might personalize and reinforce 
learning during MAKE. Early teacher input indicated that this modification was necessary, 
particularly in classrooms with uncertain staff ratio support, where “pull-out” is not feasible. 

Therefore, the LRA Greenhouse was designed to primarily serve as a Tier 1+ literacy and 
behavior regulation focused program, in which universal screening and high-quality instruction 
underlie decision-making around differentiated and individualized child supports within the 
regular education classroom. In the absence of widespread adoption of a tiered prevention- 
based system, time constraints and inconsistent (or absent) staff support, limits in specialized 
knowledge to properly lead group “intervention” (what could be done by whom), and concerns 
about early stigmatization, teachers required clear steps for intensifying their supports (who 
monitors the success of these efforts through ongoing evaluation of learning progress). In this 
way, the LRA Greenhouse supports ongoing quality improvement processes focused on 


addressing early literacy learning needs. 
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Finally, LRA Greenhouse literacy instruction also makes use of repeated “cycled targets” 


(Kaderavek & Justice, 2004). This type of horizontal approach involves simultaneous, multiple 


goals instead of vertical mastery of skills. The repeated cycling of multiple learning goals across 


months served to ease teacher “uptake” of the instructional plan. It also created regular literacy 


routines for children with different options for reinforcing learning, to provide variety for 


distributed practice (Archer & Hughes, 2011). How main literacy skills were incorporated into 


daily literacy activities each month is displayed in the table below. 


Literacy Activities by Day and Skill (Repeated Each Unit) 
PA-Beg Sound Letter Sound pringconcente Critical ieneaae 
Awareness Knowledge P Thinking amar 
Circle, Story Story Story 
Make Make 
Day 1 Play Play Play 
Do Do 
Circle, Story Circle, Story Story 
Make Make 
Day 2 Play Play Play 
Do Do 
Circle, Story Circle, Story Story Story 
Make Make 
Day 3 Play Play Play Play 
Do 
Circle Circle, Story Circle, Story Story 
Make Make Make 
Day 4 Play Play Play Play 
Do Do Do 
Circle, Story Circle, Story 
Make Make Make Make 
Day 5 Play Play Play Play 
Do Do 


Note. Italicized (Make, Play, Do) activities are small-group ENGAGE center activities. Circle = 


EXPLAIN. Story = EXPLORE. 
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Each unit repeats the same activities each month for a different targeted beginning 
letter-sound. However, PLAY themes vary in two main ways within and across literacy units. 
Within each unit, the design of PLAY emphasizes beginning sound recognition through the use 
of non-print materials on Days 1 and 2 (e.g., a “recipe” might use images only). On Days 3 and 
4, print is introduced (e.g., the “recipe” might display simple words, with the beginning target 
letter highlighted, along with images). To maintain children’s PLAY interest, two novel themes 
are employed within each unit (e.g., “pizza parlor” and “pet” store for “/p/ + Letter P”). 

Behavior Regulation Units. The focus of Behavior Regulation units shifts across four 
consecutive days (with the teacher “putting it all together” in their own way on Day 5) and each 
unit is organized around a monthly theme. Instructional targets “unfold” daily within a theme 
and are later repeated, where appropriate, in later units. Across six units, key concepts are 
introduced to help children learn mindful emotional awareness and coping strategies, different 
roles and routines within their community and what they can do (to build task and behavior 
regulation awareness), and social-emotional and task engagement regulation for problem- 
solving and self-control; concepts build on each other and culminate in their application toward 
becoming Kindergarten ready in the final unit. 

Similar to Literacy units, both “prompted” and “instructional” intentional practices are 
used for instructional balance. Activities were drawn from the What Works Clearinghouse and 
influential early childhood centers, such as Center on the Social and Emotional Foundation for 
Early Learning (CSFEL); Promising Practices Network (PPN); and Center on Enhancing Early 
Learning Outcomes (CEELO); and Collaborative for Academic, Social, and Emotional Learning 


(CASEL). In addition, activities were drawn from prior mindfulness research conducted with 


48 


children (e.g., Flook, Goldberg, Pinger, & Davidson, 2015; Schonert-Reichl, et al., 2015; Zelazo & 
Lyons, 2012). We also gained inspiration from internet and library searches, as well as early 
childhood teacher websites, for related themes and storybook selections. 

Although Behavior Regulation unit activities follow the EXPLAIN, ENGAGE, and EXPLORE 
structure, one important difference was incorporated for PLAY activities, based on field-testing. 
To achieve targeted behavior-regulation learning goals, instead of dress-up, a partially scripted 
picture card storytelling activity was developed to engage children in imaginative play. We 
chose storytelling for PLAY because, like socio-dramatic play, it is a child-led activity that can 
support literacy development as well as vicarious self-expression and problem solving. In 
addition, storytelling enabled us to focus on different, specific behavior regulation experiences 
each day (akin to the focus on particular alphabet letters for PLAY within literacy units). For 
example, all stories were carefully written about “Buddy”, the monkey children encounter on 
the LRA WM span task, who is a Preschooler transitioning to Kindergarten. Buddy experiences 
highly relatable problems to help children explore behavior regulation challenges linked to each 
daily topic and related instructional goals. 

In the initial version of this activity, the story was introduced during PLAY. However, 
based on field-testing classroom observations, we added teacher modeling of storytelling 
during Circle to strengthen the quality of children’s later storytelling (with or without adult 
support). That is, teachers model daily storytelling using the open-ended story script 
(approximately 100 words) and assigned picture cards. Later, during PLAY, children tell their 
own version of the same story using the picture cards (and teacher assistance, as available). 


This design change led to improved story richness and organization during PLAY, and better 
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supported assisting staff who facilitate the activity. Because teachers make up their own story 
ending, we found that it also provided teachers with the opportunity to emphasize points they 
wanted to share. 

Across the six units, final behavior regulation concepts and skills were selected and 

organized to help children: 

(1) Develop awareness about how their Big 4 Feelings (happy, mad, sad, and afraid) 
look and feel, expand their vocabularies for describing different feeling intensities, 
and regularly practice evaluating feelings, 

(2) Learn and practice three general Helpful Choice strategies for calming down 
(breathing, talking, and relaxing, which teachers explore using their own specific 
methods with curriculum guidance), 

(3) Develop awareness about “jobs” (tasks) and routines that are done as part of 
“Community Caring”, 

(4) Explore and practice team problem solving (and the emotional and behavior 
attributes involved for handling challenges), 

(5) Extend Helpful Choice use to task-related challenges for supporting self-control, and 


(6) Revisit key concepts with specific reference to the Kindergarten transition. 


We focused on melding children’s developing knowledge around emotions (and 
emphasized four specific feelings) to simplify learning, build mindful awareness about feelings 
and behaviors, and practice using helpful strategies for managing intra-personal, social, and 
task challenges. Theme selection was based on the perspective that Preschool children benefit 


from learning life skills for self-managing their behaviors (i.e., self-regulation; Raver, et al., 
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2011; Skibbe, Montroy, Bowles, & Morrison, 2019), particularly in contrast to having behaviors 
solely managed by teachers (Bilmes, 2004). Importantly, the Helpful Choices strategies 
encourage teachers to use (or find) specific methods that work because they are embedded 
within their classroom ecology and can be implemented outside of Greenhouse activities. 
Although general suggested techniques are provided for teacher inspiration, instructional 
details are not provided in the curriculum. 

Another unique feature of the Behavior units is that children engage in a daily routine of 
emotional self-evaluation using the Big 4 Feelings concepts learned in the first unit. For 
example, teachers create a classroom pocket poster to give children a structured opportunity 
for evaluating their feelings in particular any moment. To provide structure, prior to Circle, on 
each Behavior unit day, children are asked to “notice” how they are feeling and place a Big 4 
Feeling emoji card in their pocket to show it. Teachers are trained to use this opportunity 
throughout the day, but at least once a day, to give children practice noticing and expressing (if 
they desire) how they feel. In addition, two concepts (“glitter mind jars” and a “beginning, 
middle, end” feelings process) are used across Behavior units to convey the transient nature of 
emotions and how we can control their intensities. 

Below we show the five main areas emphasized for each unit. We note that 
“Community Caring” and “Team Problem Solving” units also emphasize teaching of routines, 
task engagement, and teamwork, which involved learning about community helpers, as well as 
exploring how children can help their community, topics that were outside of the main areas of 


our focus but strongly grounded in early childhood and kindergarten readiness practices. 
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Behavior Regulation Instructional Targets by Thematic Unit and Activity 


Mindfulness & Self 


Behavior Unit Emotions Awareness Behavior Concepts Critical Thinking Language 
Knowledge 
EXPLAIN**** EXPLAIN **** EXPLAIN **** EXPLAIN **** EXPLAIN **** 
Unit 1 Make**** Make**** Make**** Make**** Make**** 
“Big 4 Feelings” Play **** Play**** Play**** Play**** Play**** 
Do**** Do**** Do**** Do**** Do**** 
EXPLORE**** EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** 
EXPLAIN *** EXPLAIN ** EXPLAIN **** EXPLAIN **** EXPLAIN **** 
Unit 2 Make*** Make** Make** Make ** Make* 
“Worked Up Feeling | Play*** Play** Play**** Play**** Play**** 
Choices” Do** Do** Do*** Do*** Do** 
EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** 
EXPLAIN ** EXPLAIN *** EXPLAIN **** EXPLAIN **** EXPLAIN **** 
Unit 3 Make** Make*** Make* 
ZcommunlepCanag’ Play** Play*** Play **** Play**** Play **** 
Do*** Do**** Do**** 
EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** 
EXPLAIN *** EXPLAIN * EXPLAIN **** EXPLAIN **** EXPLAIN **** 
Unit 4 Make** Make** Make*** Make* 
“Team Problem Play*** Play* Play**** Play**** Play**** 
Solving” Do**** Do** Do*** Do** 
EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** 
EXPLAIN **** EXPLAIN **** EXPLAIN **** EXPLAIN **** EXPLAIN **** 
Unit 5 Make***** Make*** Make*** Make**** Make ** 
“Self-Control” Play **** Play**** Play**** Play**** Play**** 
Do*** Do** Do**** Do**** Do*** 
EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** EXPLORE **** 
EXPLAIN **** EXPLAIN ** EXPLAIN **** EXPLAIN **** EXPLAIN **** 
Unit 6 Make*** Make ** Make ** Make*** Make ** 
“Kindergarten Play**** Play** Play**** Play**** Play**** 
Ready” Do*** Do*** Do** Do**** Do*** 


EXPLORE *** 


EXPLORE **** 


EXPLORE **** 


EXPLORE **** 


EXPLORE **** 


Note. * = # of days of emphasis per unit (out of 4 possible). Italicized (Make, Play, Do) activities = ENGAGE center activities. 
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Progress Monitoring 

An activity checklist feature was developed as a “pop-up” screen at key locations within 
activity plans. Prior research has documented the challenges that teachers experience when 
monitoring progress in the classroom (Roehrig, et al., 2008). Furthermore, perceived usefulness 
greatly shapes users’ acceptance of technology (Davis, 1993). Consequently, the checklist app 
feature provides teachers with a simple, organized system for documenting how difficult 
learning new concepts and behaviors are for children in their class, particularly those identified 
as having “High Priority Needs”. Through evaluating and reporting whether the activity was 
easy, needed support, or hard (with support) across instructional days, teachers develop a 
clearer understanding of individual children’s learning strengths and needs, and how they 
change (or not) over time. In addition, a text box space is provided for teachers to reflect and 
report specific observations about whether behavior interfered with learning for a particular 
activity. Information can be inputted across multiple activities within a Literacy unit day, and 
across units, to help teachers pinpoint potential patterns (when examining the output in the 
Activity Checklist report) that impact a child’s learning so that they can better hone their 
individualized supports for children over time. 

We worked closely with teachers to design the interface, location, and amount of 
informational input for monitoring children’s learning progress. The adoption of educational 
innovations and new practices in support of reading development is hampered when perceived 
by teachers as time-intensive (Al Otaiba, Hosp, et al., 2008; Roehrig et al., 2008). Although 
progress monitoring may bring instructional focus, it may also create challenges with 


technology use (input and output of information), information use, and time management 
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(Roehrig et al., 2008). Therefore, we were careful to design the checklist app feature to enable 
teachers to efficiently document “in the moment” how learning proceeded for children across 
three basic levels (easy, needed support, or hard with support) and whether they perceived 
either internalizing or externalizing behaviors as impediments to learning within a specific 
activity. We include behavior with literacy skill monitoring because of their reciprocal effect on 
learning (Coie & Krehbiel, 1984; Goble et al., 2017; Morgan, et al., 2008). We included the text 
box feature out of teachers’ expressed desire for recording more extensive observations about 
skill, behavior, or implementation factors (e.g., activity adaptations made). A report was also 
created (described below) to help teachers evaluate progress over time by pinpointing specific 
patterns of strength and weakness across literacy learning classroom activities. See screenshot 


below. 


Activity Plan 


Homes Meeced 


Implementation Resources 
To foster sustainable use of the LRA Greenhouse, a variety of implementation resources 


(35+) were developed to quickly and easily scaffold teacher use. During the first two years of 
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development, we conducted in-depth teacher interviews to learn about teachers’ perceptions 
concerning what worked well and less well in their classrooms, as well as implementation 
strengths, needs, and barriers. We asked structured, regular questions to examine changes over 
time, as well as made specific inquiries based on observed implementation. We also used 
teacher feedback, comments, and questions as sources for developing intentional resources. 

Based on our findings, we created resources to meet three kinds of needs: teacher 
knowledge building, instructional preparation, and activities implementation. Tool use requires 
content, pedagogy, and technological knowledge (Mishra & Kehler, 2006) for effective 
assessment-guided instructional decision-making activities. Teachers’ feedback clearly 
emphasized their lack of time for reading a manual, and their desire for more manageable, 
“bite-size” chunks of information. Therefore, we created a number of technological, 
assessment, and instructional resources that could be used “on the fly” to provide teachers 
with content, pedagogical, and technological knowledge efficiently. We also created paper and 
video resources to aid in preparing to implement the LRA Greenhouse to help them develop 
confidence and early success (Ertmer & Ottenbreit-Leftwich, 2010). 

For example, based on a composite created from Prekindergarten teacher experiences 
of learning how to use the LRA Greenhouse, we created brief “think aloud” videos for enabling 
teachers to witness another teacher’s early installation steps. We created “Table Topper” 
supports for keeping ENGAGE center activities connected, organized, and scaffolded for multi- 
teacher facilitation. In addition, we created word lists to help teachers engage in authentic yet 
strategic dialogues with children about the monthly letter-sound. Working memory load (i.e., 


how much teachers are mentally processing) is positively associated with language production 


55 


(Belke, 2008); therefore, we aimed to support authentic, yet strategic, conversation by listing 

common words for teachers to reduce mental demands. Teachers reported that this resource 
not only helped them “think of words to say” for intentional talk, but also more quickly gather 
materials in preparation for implementation. 

We also created two types of letters to be sent home in order to enhance 
communication about the LRA Greenhouse for families. One letter introduces the program and 
explains some instructional differences that may be observed on LRA Greenhouse days, and the 
other is a series of letters (one is sent home each month) to encourage families to bring in 
items that “begin with the monthly letter”) to encourage family participation on Day 3 literacy 
learning activities. The letter provides brand examples and a return slip response option to let 
teachers know what might be brought in to support the letter recognition “pass around” 
activity. Family involvement in early childhood learning is an important component of quality 
programming because it: enhances communication, builds teacher-family connections, creates 
volunteer opportunities, and strengthens collaborative decision-making (Morrison, et al., 2015). 
We aimed for our Day 3 environmental print activity to encourage parent volunteering to 
promote classroom literacy learning connections, as well as pride among children for their 
family’s contribution to the lesson. In the tables, below, we list these implementation resources 


available and their intended purposes. 
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Knowledge Building 


Implementation Resource 


Purpose 


Key Terms Glossary 


Defines essential words used in the LRA 
Greenhouse system 


16 “How-To” support 1-pagers 
e Navigate the Tablet App 
e Navigate the Training 
e Navigate the Support Website 
e Create the Class List 
e Administer the LRA (2 Checklists) 
e Complete the Behavior Rating Scale 
e Use the Activity Plan 
e Use the Activities Checklist 
e Read Reports & Interpret LRA Scores 
e Prepare the “How Do | Feel? Poster 
e Prepare Behavior Unit Story Picture 
Cards 
e Make Glitter Mind Jars 
e Make the “Same Sound Game” 
e Use Environmental Print 
e Choose a Story 


Provides quick “on the go, need to know” 
support for technology use, key activity 
materials preparation, and making informed 
decisions that are aligned with the LRA 
Greenhouse approach 


Preparation 


Implementation Resource 


Purpose 


Dramatic Play Theme Overview 


Provides a preview of the dramatic play 
themes to help with early prep and gathering 
of materials 


Extended Suggested Story List 


Offers other, suggested stories (for extended 
for substitution use) for LRA curriculum units 


LRA video walk-through 


Demonstrates a quick overview of different 
assessment parts 


5 brief “Think Aloud” videos & 1 “Getting 
Started Literacy unit organizer 


Provides a teacher model for organizing, 
preparing, and coordinating LRA Greenhouse 
system use, based on insights gleaned from 
teachers who helped with the tool’s 
development 


Family Introduction to the LRA Greenhouse 
Letter 


Introduces families to the classroom use of 
the LRA Greenhouse and explains some 
potential differences in typical activities 
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Activities Implementation 


Implementation Resource 


Purpose 


Curriculum Printables 
e Word Lists 


e 54 1-page activity “Table Toppers” for 
MAKE, DO, and PLAY 
e 6 Family Letters Home 


e Supports teachers’ use of authentic + 
intentional teaching language 

e Provides the activity goal, directions, 
and suggested prompting 

e Informs and encourages family 
participation on Day 3 during the 
Literacy Circle activity by bringing 
labeled items in 


Ensuring Children’s Success 


Describes what “successful learning” looks 
like when using the LRA Greenhouse to guide 
instructional support decisions 


Implementation Fidelity Checklist 


A 5-day instructional observation checklist 
for helping administrators evaluate the 
quality of key literacy unit strategies 
implementation. 


Below, is screenshot of example resources (from top left: Table Topper, Literacy Unit 


Family Letter Home, Word List, and Activity Plan suggested phrasing). 


oa 


BC 
Make a Letter B story 


A Make Table Activity Support A 
Baby Bee Book 


Materials and Resources Needed 


Suggested Intentional Phrasing 


Every story has a beginning, middle, & end, Let's create a story together about BES and BOB Butterfly. Each of you will get a picture 

card with something they might do, see, think, or feel in our story. When it's your turn, you will add your picture card idea to our story. | 
will think of the story's beginning, and you will help make up the middle. Then, | will add the end, and after everyone has had a turn, we 
TITLE) My picture is. 
had never crossed the bridge to see what was on the other side of the ocean. One day... (prompt first child to begin using a ¢ 


will read our story aloud! (REAL 


A B CDEFGHTJKLMNOPQRST UVWXYZ. 


Letter B words that begin with the /b/ sound 


In Home or 
Community 


now): BEACH, Once upon a time, BES and BOB lived on the beach, but they 
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Wrap-Around Teacher Supports 

Planning Calendar. We created monthly planning calendar to help teachers organize 
implementation. Because sites differ in their daily routines (e.g., due to fieldtrips or regular 
community outings), and to facilitate timely assessment and activities completion with regular 
follow-through, the calendar was designed as a gateway to Activity Plans. This feature displays 
days colored by unit (or assessment window), and allows teachers to type on LRA Greenhouse 
days and uncolored “own days” (in which teachers implement their own activities). The 
monthly view is printable, allowing teachers to share instructional goals. In addition, on “own 
days,” teachers can select from aligned literacy themes (e.g., “space” during the /s/+ Letter S 
unit) or type in their own ideas. To facilitate tracking implementation teachers can also mark 
daily activities as complete on the calendar. Gold coloring represents assessment days, pink 
coloring represents Behavior Regulation unit instructional days (lighter pink for “own day”), 
green coloring represents Literacy unit instructional days (lighter green for “own days”), and 


white coloring represents non-Greenhouse “own days”. See calendar screenshot below. 


Greenhouse Plan 


Previous March 2018 


Assessment 


5 6 7 8 
Behavior Day 1: Expicting | Day 2: Expiaring Day 3: Exploring Day 4: Exploring 
Working Mand Somng TEAMWORK CHALLENGES BRAVERY & HELP-SEEKING 
Protons DETERMINATION 
12 1 
Literacy Own Day Own 


20 22 
Literacy Own Day Own Day 
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Calendars can provide not only “external memory aid” for teachers (McDonald, et al., 
2011), but also support for organizing information. For example, we aimed for a clearly 
organized calendar to visually facilitate “coordination” between LRA Greenhouse instructional 
days and teachers’ “own days”. Because teachers can toggle the monthly view, they can 
prepare and plan for the coming month or review previous the month simply and quickly. 

Support Website. We developed a companion website to create an internet-based 
structural “container” for various program elements that provides teachers with scaffolded 
information (e.g., photos of activity set-up, reminders), access to the tablet app, training, and 
printable materials and resources. We anticipated the website would facilitate tool use by 
keeping information easily accessible, connected, and simply organized. Conceptualized as a 
“greenhouse” where teachers can access tools to promote literacy growth, the site included 
both public and private (registered users-only) materials. Of the drop-down menu options, 
three (Printables, App Access, and Training) are private. The Printables drop-down tab includes 
a “Get Ready” page that contains images, need to know resources (guidance documents, family 
introduction letter, and HOW-TOs), and brief teacher “think aloud” videos for getting 
organized. Access to the app allows teachers to navigate the tablet app on a computer so they 
can flexibly alter their class list, complete the behavior rating scale, and view/print LRA reports. 
As noted by McLoughlin and Marshall (2000), “scaffolding is intended to motivate the learner, 
reduce task complexity, provide structure and reduce learner frustration”. Through iterative 
work with teachers, we developed a website that functioned to scaffold LRA Greenhouse users, 
so that it benefitted teachers motivationally, educationally (in terms of facilitating their 


learning), and logistically. See screenshot below. 
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LRA Greenhouse 


Growing Recepuve Learners for Reading Success 


Week t: Days 1 & 2 Beginning Sound /b/ Printattes 
¢ Did you send the monthly letter home to farnilies? 

¥ Did you gather EXPLAIN examples & non-exampies for Circe? 
¢ Did you find the recommended stories for EXPLORE? 


© Days 1 & 2 Black and White print bungie 


For trese activities print and game’ me folowing: Same Sound containers colors. glue scissors 


LRA Score Reports. We worked closely with teachers to enhance the assessment results 
information available for making intentional teaching and support decisions to prevent RD. An 
important aspect of our work focused on balancing the amount and kind of information to keep 
results accessible, meaningful, and actionable to prevent “information overload”. The LRA 
Greenhouse tool provides teachers with four assessment score reports (see screenshot 
examples below, presented clockwise beginning with the top left): 

(1) Class Scores Report for seeing how all children in a class are doing, 

(2) High Priority Needs (HPN) Report for seeing which children have special learning 

needs to address, 

(3) Activity Checklist Report for monitoring the progress of children on literacy 

activities (especially those with HPN), and 


(4) Individual Score Report for seeing how each child is doing across the year. 
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Each report provides different information to help Prekindergarten teachers make 
“assessment-guided” decisions for providing supports to children who need it. Although some 
teachers may be able to early “identify” children at risk for learning problems (Taylor et al., 
2000), using LRA results in conjunction with the curriculum activities can help all teachers 
pinpoint where individualized needs can be best met, which research shows is an effective 
practice for addressing learning difficulties by promoting growth (Chard, et al., 2008; Connor, et 
al., 2006; Fuchs & Fuchs, 2006; Hamilton, et al., 2009; vanDerHeyden et al., 2008). 

Teachers are trained to first review the Class Scores Report to gain an understanding of 
overall class functioning, and then the HPN Report, which identifies specific children whose 
performance on the LRA suggests immediate support needs. After activity completion (and 
teacher input), the Activity Checklist Report can be examined to understand in-classroom 
learning patterns to help teachers tailor their supports and make stronger assessment-guided 
decisions to reduce children’s risk for RD. Finally, the Individual Score Report is a printable 


report for sharing individual growth over time with parents and other practitioners. In addition 
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to an online training module dedicated to understanding the LRA and score meaning, teachers 
can also access resources to help them effectively reflect on how to meaningfully evaluate and 
act upon LRA results. Beyond making instructional changes based on teachers’ interpretation of 
assessment results, teachers also engage in a feedback loop about their teaching that can 
positively impact their practice (Van den Hurk, Houtveen, & Van de Grift, 2016). 

Online Training. Through iterative development cycles involving teacher feedback and a 
“think aloud” protocol, we created six online training modules that covered the breadth and 
depth of LRA Greenhouse content and implementation. We aimed for the online training to be 
high-quality, useful, and relevant for teachers. An important aspect of this training was to 
enhance teachers’ understanding about their instructional role in organizing children’s 
knowledge (Archer & Hughes, 2011). Our secondary aim was for the training modules to be 
teacher-friendly and time-sensitive, so that successful completion was feasible. 

The training was designed to cover tool and curriculum fundamentals: Literacy 
(phonological sensitivity, sound to letter mapping, and meaning-making) and Behavior 
Regulation facts, procedures, and concepts (emotion awareness and labeling, mindful calming 
strategy selection, self-control, task and prosocial engaged behaviors) for effective curriculum 
activities implementation, and assessment administration, tool use, and assessment-guided 
instructional decision-making. Although the content differs by module, we implemented 
consistent design features taken from best practices in online professional development to 
facilitate teacher engagement and consistent learning strategies (e.g., Clark & Mayer, 2003). 
For example, we embedded and interspersed 10 multiple-choice quiz questions throughout the 


modules, with either validating (for correct selections) or guiding (for incorrect selections) 
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feedback, to provide checkpoints for learning. We aimed to equip teachers with sufficient, 
rather than exhaustive, grounding to effectively deploy the 5-step model and strategies. Our 
goal was to establish a firm foundation to allow additional “on the job” professional enrichment 
through continuous formative data use learning (Mandinach & Jimerson, 2016). 

Each module was designed to take 30- to 60-minutes. Teachers who successfully 
complete the training earn six Continuing Education (CE) credits through the Oregon Registry (a 
printable completion certificate is available once an overall passing score is achieved). Low 
scoring modules can be retaken after a 6-hour delay. Six teachers were surveyed about their 
ease of completing the training using a 1 (hard) to 5 (easy) scale. On average, teachers rated 
their experience favorably, M = 4.17, and they agreed that completing the training helped them 
to more effectively implement activities (MW = 3.83; 1 = strongly disagree to 5 = strongly agree). 

Through online training, we focused on enhancing five main assessment-guided steps 
through teachers’ use of the LRA Greenhouse: (1) implementing high-quality assessment 
practices; (2) using assessment information to strategically advance literacy growth for children 
with High Priority Needs (RD Risk); (3) implementing high-quality intentional teaching using 
evidence-informed practices, (4) systematically monitor learning to promote literacy skills 
development; and (5) engage in ongoing reflection and evaluation of literacy growth and RD 
risk needs. Teachers need to be able to effectively screen for RD and understand the purpose 
for LRA use before they can meaningfully use assessment results to support diverse literacy 
learning needs, particularly when the assessment method is unfamiliar to the field. In addition, 
teachers need to use strategic and high-quality instructional practices (including systematic 


monitoring of literacy learning progress) to help children develop critical pre-reading skills. 
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Finally, they need to engage in ongoing reflection about the children at-risk for RD and how 
their difficulties manifest in their classroom, so that they can evaluate their support and make 
timely and differentiated adjustments, as needed (e.g., Hoover, 2011). Below we show a 


screenshot of the training modules portal. 


| 
LRA Training Site Training 


As : 
Module 6: Learning Supportive 


Behaviors 
Incomplete 


These mutimedia training modules provide training on the acministration of the learning actties and assessments provided by the LAA 
oe 
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LRA Greenhouse Pilot Study Findings 


We engaged in two pilot studies to examine LRA Greenhouse functioning. In the first 
pilot study, we compared LRA performance between six-month implementing and non- 
implementing classrooms. In the second pilot study, we examined the impact of partial LRA 
Greenhouse implementation (across three-months). We report quantitative findings from both 
studies, followed by teacher survey and interview findings that describe the tool’s impact. 
Pilot Study 1 
Sample 

Prekindergarten Participants. The initial pilot study of the full LRA battery with 
Greenhouse curriculum use was conducted in two waves across four Prekindergarten 
classrooms (n = 37 children). All classrooms were taught by Caucasian female teachers, ranging 
in age from 35 - 55 years, with 10 to 35 years of experience located within one mid-size Oregon 
city (median regional household income = $50,592). Sites included two center- and two home- 
based settings (reflecting a mix of four- and five-star ratings, out of five possible, within 
Oregon’s Quality Rating for Improvement System [QRIS], now Spark in Oregon; 
https://oregonearlylearning.com/spark). On average, children were 53.5 months old in the fall, 
were primarily Caucasian (88.6% Caucasian, 8.6% Latino, and 5.4% Asian) and 21.6% had 
identified special needs pertaining to concomitant English Learning and developmental 
disabilities (cognitive, motor, visual, language). Forty-six percent of the children were female. 

Class size ranged from 10 to 18 children, with a minimum of 6 Prekindergarten children 
per class (children younger than four were excluded from analyses). Teachers used published 


curricula in their daily practice (e.g., Creative Curriculum or High Scope) or a “classroom 


66 


created” curriculum (using materials either gathered from the internet or self-made) consistent 
with their unique educational philosophy. 

Comparison Group. We examined fall and spring growth between four LRA Greenhouse 
implementing classrooms and four (non-implementing) comparison classrooms that were 
located at the same large center. The non-implementing center had a QRIS four-star rating, and 
shared similar demographics with LRA Greenhouse pilot participants, although located ina 
different county. The “business as usual” classrooms comprised n = 37 children; LRA tool use 
included the completion of assessment items delivered in the fall and spring only. Classroom 
teachers were Caucasian females, ranging in age from 24 to 51 years old, with 5 to 30 years of 
preschool teaching. On average, children in these classrooms were 54.1 months old in the fall, 
and represented 81.1% Caucasian, 13.5% Latino, 2.7% African American, and 2.7% Asian 
backgrounds. Two children had identified exceptional needs (5.4% of the sample), with one 
child identified as an English Learner and another identified with a motor developmental 
disability. 

Staff support in varied from one to two adults per classrooms with three- and four-year 
old children, composed of 18 to 20 children per class; however, only four-year old 
Prekindergarten children took the LRA. Classroom instruction was observed for one hour on 
two separate occasions. Teachers used a “classroom created” developmental curriculum (using 
materials either gathered from the internet or self-made) that were considered “generally” 
grounded in High Scope principles. Instructional approaches varied between classrooms based 


on each teacher’s educational philosophy. 
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Procedures: LRA Administration 

Assessment in LRA Greenhouse implementing classrooms was observed for one hour 1 - 
2 times per seasonal occasion (most of the child-administered testing was observed within this 
time frame). Assessment for comparison classrooms was conducted by trained research staff 
and involved the same materials and procedures. Children completed Pre-reading and WM 
tasks in approximately 13 to 16 minutes each administration. Depending on site resources, 
children took the LRA using headphones and visual dividers, either individually (at the back of 
the classroom during assigned quiet times) or in small groups of four children in a separate 
location outside of the classroom. All children were given brief instructions and screen-touch 
feedback prior to beginning. When needed, administrators provided direct hand feedback to 
help children understand whether their touch pressure was “too hard, too soft, or just right.” 
Teachers completed the BRS separately on the tablet within the same assessment time frame, 
on average, in approximately 23 to 26 minutes. 

Structured observations of LRA administration fidelity conducted by research staff 
indicated that, on average, trained teachers provided children with a quiet (63% of the time), 
non-distracting (71% of the time), and comfortable (96 - 98% of the time) assessment 
environment. Resource constraints (e.g., additional staff support or space limitations) were 
found to underlie the relatively lower fidelity for the noise and distraction factors. For example, 
intermittent noise and staff/parent interruptions were noted as affecting fidelity ratings, 
despite initially sufficient-quality conditions. Effective touch-screen submission was the most 
frequently observed challenge among children, particularly for those without prior cellular 


phone or tablet experience. 
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Teachers were rated as competent administrators across time points: 95% appeared 
comfortable leading testing sessions, 97% adequately assisted children during testing (e.g., with 
submitting responses when needed), 94% adequately managed off-task behaviors (e.g., 
children talking to each other), and 95% efficiently began and ended sessions. Total fidelity 
scores were based on a 0/1 (not observed/observed) scoring using a 11-item observation 
checklist that focused on testing environment and teacher behaviors throughout the testing 
administrations. 

Procedures: Instructional Implementation 

Participants used LRA Greenhouse activities (i.e., assessment, instruction, strategic 
monitoring and support of children with High Priority Needs, and reflective ongoing evaluation 
of progress) nine days per month over six months from fall to spring. Prior to implementation, 
teachers completed the six online training modules at their convenience over 4 - 6 hours. 
Instructional implementation was observed for approximately one hour, 2 - 3 occasions per 
month. We gave limited feedback to teachers, providing clarifications when needed and 
positive appreciation for study participation. Following observations, we interviewed teachers 
using a standard question protocol to gather information about their experience. 

Teachers’ implementation of literacy activities fidelity, as measured across monthly 
occasions per teacher, depended on the activity. For example, teachers properly implemented 
center MAKE and DO center activities 92.5% and 87.5% of the time, respectively, whereas PLAY 
and circle EXPLAIN activities were implemented with fidelity 76.2% and 74.3% of the time, 
respectively. Research staff used a structured checklist with a rating scale of 0 (not observed) to 


3 (ideal implementation based on design) for evaluating multiple factors within each activity. 
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For example, levels of implementation were delineated for each of the steps within EXPLAIN 
activities as well as teacher instructional, quality of materials, and child engagement 
expectations during the other activities (based on unique daily learning goals). Total fidelity 
scores were obtained by summing activity implementation scores plus credit for using the 
tablet app (instead of printed screenshot copies, which were also available). 
Results 

Between-group mean fall Pre-reading scores were non-significant (F = .61, p = .44), 
suggesting similar skills at the beginning of the school year. Implementing and non- 
implementing comparison groups obtained mean scores of about 11 and 10 (M = 10.79 and 
10.08, SD = 4.01 and 4.25, respectively). Seventeen children in implementing classrooms 
(45.9%) and 21 children (56.8%) in comparison classrooms were identified with RD risk based 
on performance below the fall cut-point. 

By spring, mean Pre-reading scores were significantly stronger among children in the 
LRA Greenhouse implementing classrooms than children in comparison classrooms (F = 11.91, p 
< .001). More specifically, children within implementing classrooms obtained a mean spring 
Pre-reading score of 14.97 (SD = 4.57) whereas children in comparison classrooms obtained a 
mean spring Pre-reading score of 11.08 (SD = 4.83), slightly above the RD risk cut-point (see 
Appendix, Table 1). Only 5/17 children (13.5%) in LRA Greenhouse classrooms maintained their 
fall RD risk status in the spring; in contrast, 18/21 children (48.6%) in comparison classrooms 
remained at RD risk in the spring. For implementing classrooms, we found 32.4% reduction in 
RD risk with 54 days of instruction across six months; we found an 8.2% reduction in RD risk for 


non-implementing classrooms across the same time period. 
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In addition, between-group differences for WM and the BRS were examined. Non- 
significant differences between the groups in WM and task engaged behavior ratings were 
found in the fall (F = .10, p = .76, and F=1.95, p =.17, respectively), suggesting comparable 
incoming levels. By spring, significant between-group differences favoring LRA Greenhouse 
implementing classrooms were found for working memory span (F = 5.30, p = .02), but not task 
engaged behavior ratings (F = .33, p = .57). Because fall prosocial engaged behavior ratings 
were significantly different between the two groups (F = 4.31, p = .04), a spring comparison for 
this measure was untenable (see Appendix, Table 1). 

In summary, we found important significant between-group Pre-reading differences for 
LRA Greenhouse implementing classrooms, in which children increased their literacy scores, on 
average, four points. Significant gains in WM span were also found for LRA Greenhouse 
implementing classrooms, when compared to “business as usual” classrooms. Children in both 
groups made beneficial gains, on average, in behavior regulation; however, the differences 
between groups was less clear. 

Pilot Study 2 

We also examined performance when the LRA Greenhouse was partially implemented 
(for three, instead of six, months) to see whether similar trends existed. In addition, we 
recruited more diverse sites to evaluate the generalizability of our Pilot Study 1 findings. 
Sample 

Prekindergarten Participants. The partial implementation sample (n = 24 four-year old 
children in four classrooms) comprised 87.5% Caucasian, 8.3% Latino, and 2.9% Pacific Islander 


backgrounds. On average, the children were 52.4 months old at Time 1 (in the Winter). Forty-six 
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percent were female, and 6% of children had identified special needs; one pertaining to 
concomitant English Learning and one related to motor developmental delay. 

All classrooms were taught by Caucasian female teachers, ranging in age from 30 - 50 
years, with 5 to 30 years of experience across mid-size Oregon city (median household income 
range across two cities = $50,592 - $56,186) and rural town (median household income = 
$49,835) locations. Sites included two center- and one home-based locations (reflecting greater 
diversity in quality rating and locale than Pilot Study 1: one 4-star metropolitan, one 3-star 
rural, and one 5-star suburban site). Class size ranged from 10 to 20 children, with a minimum 
of 4 Prekindergarten children per class (children younger than four were excluded from 
analyses). Teachers used published curricula in their daily practice (e.g., High Scope) or a 
“classroom created” curriculum (using materials either gathered from the internet or self- 


made) consistent with their unique educational philosophy. 


Procedures 

Winter and Spring LRA assessment was conducted by either the teacher, with research 
team support, or completely by the research team. Similar to Study 1 participation, 
assessments were administered before and after the commencement of instruction. Literacy 
and Behavior unit activities were implemented as previously described, sans three Literacy and 
Behavior units. The removed units were taken from the middle of the year so that instruction 
began with the same first four foundational units (i.e., two each of Literacy and two Behavior 
Regulation) and ended with the same unit for both Pilot 1 and 2 participants. Fidelity of 
instruction data were not collected. However, teachers engaged in the same training as 


described in Pilot Study 1 and similar classroom observation procedures (in terms of duration 
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and frequency). In general, teachers implemented the activities as intended, and similar to (but 
less robustly than) Pilot Study 1 participants. 
Results 

On average, children began with Pre-reading mean scores of 11.04 (SD = 3.29) in the 
winter, slightly above the risk cut-point. Similar to Pilot Study 1, children generally improved in 
all areas measured after 3 months (27 instructional days) of LRA Greenhouse use. Gains in 
mean Pre-reading scores for the spring were found (M = 12.58, SD = 3.51), which were 
significantly different than Time 1 in the winter (t = -2.80, p = .01). Gains were also found for 
WM (M = .75, SD = .79 and M = 1.04, SD = .75 for Times 1 and 2, respectively) and both parts of 
the BRS (Mask engaged = 49.71, SD = 9.54 and M = 50.33, SD = 8.84 for Times 1 and 2, respectively; 
Mprosocial engaged= 52.13, SD = 8.78 and M = 54.33, SD = 8.46 for Times 1 and 2, respectively). 
However, gains were statistically non-significant (twm = -1.50, p = .15; ttask engaged behavior = -0.45, p 
= .65); prosocial engaged behavior regulation gains approached statistical significance (t = -1.88, 
p =.07). See Appendix Table 2 for all reported scores. 

Importantly, Pre-reading scores for children identified with High Priority Needs RD risk 
increased 20%, in contrast to a 4.5% increase for non-risk peers, reflective of highly strategic 
and selective literacy skill building differentiation for the RD risk group. At time 1 (winter), 
nearly 63% of the Prekindergarten sample was identified with RD risk based on performance 
below the winter cut-point. By time 2 (spring), however, only 40% maintained their RD risk 


status, a 23% reduction within just three months of implementation (27 days of instruction). 
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Pilot Studies 1 and 2 Teacher Feedback 

Across pilot studies, Prekindergarten teacher users highly rated the instructional 
support and resources developed for enabling assessment-guided instructional decision- 
making. In addition, surveyed teachers (n = 6) strongly agreed (M = 4.5/5; where 1 = strongly 
disagree to 5 = strongly agree) that Activity Plans and support resources were helpful for 
implementing activities. All six teachers (100%) agreed or strongly agreed that implementing 
LRA Greenhouse activities was a valuable experience. 

We repeatedly surveyed implementing teachers 39 times over the course of their tool 
use and found that they would “recommend the activities in this unit to a friend” 87% of the 
time, and that they judged the activities as developmentally appropriate 95% of the time. In 
addition, 100% of teachers agreed that the assessment “appropriately identified” children at- 
risk for RD and that using this information “fit” with their typical approach for helping children 
develop literacy. Teachers unanimously agreed that they were able to rate behaviors in their 
class efficiently, and 75% agreed that both administering the LRA and finding test results were 
easy. Seventy-five percent of teachers agreed that the LRA Greenhouse companion support 
website was well-organized. Teachers also rated the curriculum activities as very easy to 
implement (Median rating: 5/5 for nearly 80% of surveyed occasions across six months of 
piloting). In general, teachers reported using other developed resources (e.g., “How To” one- 
pagers) on a “need to know” basis, as intended. 

The main themes that emerged from teacher interviews and survey responses (see 
Appendix, Tables 5 - 10), which illustrate teachers’ reported perceptions about the LRA 


Greenhouse’s suitability were: Feasibility, Accessibility, Usefulness, Fit, Facilitation, and Impact 
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on Practice. Feasibility was assessed across 39 occasions (between 4 - 7 occasions per teacher); 
74% of the time teachers reported no “hiccups” or challenges with implementation, were likely 
to recommend activities to a friend, and found the activities easy to implement and for children 
to learn (Table 5). In terms of accessibility, teachers reported ease of use and efficiency across 
various aspects of the technological tool (Tables 6a and 6b). In terms of usefulness, teachers 
agreed that the LRA was “useful” (Table 7): LRA results were viewed as appropriate and 
meaningful for differentiating children to meet their learning needs. In terms of fit into their 
practice, teachers unanimously agreed that LRA items were developmentally appropriate for 
children, and the results fit with their typical approaches for supporting children’s development 
(Table 8). The tool also was perceived by teachers as facilitative, and helpful for knowing who 
needed strategic literacy-learning help, planning and providing intentional instruction, 
monitoring progress, and generating new insights (about children and classroom practices). See 
Table 9. 

Finally, 75% of teachers reported an impact of tool use on their practice (Table 10) — 
identifying children who need greater support because they are at-risk for later learning 
difficulties, modifying how they help these children, and explaining their learning needs to 
others. Nearly 88% of teachers reported monitoring progress more carefully and being capable 
of discussing children’s learning needs differently with parents as a result of their LRA 
Greenhouse experience. Thus, our survey findings suggest that LRA Greenhouse use benefitted 
teacher practice and child learning over time, and enhanced teacher knowledge to create more 
lasting impacts. Below, we share quotes gathered during the teacher interview process, to 


highlight their perceptions for each of the categories using their own words. 
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Feasibility: Ease and efficiency of LRA Greenhouse implementation in preschool classrooms 


“| had everything | needed- | only bought coffee cups, and you can make it as extensive as you 
want.” 


“Is super organized and straightforward with its directions. For a teacher who is needing help 
with planning, it would make it time-efficient and easy.” 


“| liked being able to go to one spot and print off everything | needed.” 


“I really liked that some of the days the activities were planned for me. Once you get to the 
days, a lot of my job was done for me outside of the prep...lt made planning easier — it focused 
the calendar for me.” 


“(It took] about 10-15 minutes each day to set up [and gather materials]. At the beginning of a 
week, | spent 20 minutes laminating, cutting, printing.” 


Accessibility: Simplicity and straightforwardness of access to and use of LRA Greenhouse 
components 


“I think it’s been great. Really easy to use. Everything was easy to find and print out. Worked 
really well.” 


“| just pull up the website and enter it [in the tablet app] at home through the website. That is a 
really nice thing about your program, that everything is accessible on the website, as well as the 
tablet.” 


“The program was really user-friendly.” 
“I liked the way everything [in the training] was broken down to be clear.” 


“I’m not a natural with technology, but it was definitely helpful to remind me of what | need to do. 
| am better able to use it... I’ve got it now- | know how to click through it.” 


Usefulness: Effectiveness of LRA Greenhouse at characterizing children’s literacy skills and 
learning needs 


“It was great. The assessment really appealed to me because | have a program that | use and it 
has assessments, but the benchmarks were hard for me to understand and to use. The 
curriculum was great for understanding the emotions and the letter activities. It was really 
appropriate. | liked a lot of how the curriculum encouraged discussion and play-based approach 
to learning. It’s good not only so | can tweak my program, but also show parents how their 
children have grown. Simple and easy to use.” 


“I got to see if they were progressing or going backwards because you can’t [ordinarily] see that 
daily. | got to see the growth they were making. It showed me the kiddos | need to pay more 
attention to and follow up on my part, outside of the learning moment throughout the rest of the 
day.” 
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“The curriculum had it ready to implement and | had a real score for me to understand [from the 
LRA], and then there was 3 terms to work with... [to] see the progress.” 


Fit: Appropriateness of LRA Greenhouse for preschool classrooms 


“Because there were so many different activities, it helped to include the different learning styles 
of the kids and allowed them to ‘get it’ later...it gave me more specific ways to support them and 
the language | needed, which was nice.” 


“I still get to plug in my own things that | would do with my typical curriculum because of the 
days that the Greenhouse curriculum doesn’t cover every day.” 


“The [assessment] tool and activities are aligned with our goals... Fit right with what we were 
already doing.” 


“| really like the circle stuff- all of it is on point and the kids really get it all pretty quickly.” 


Facilitation: Capacity of LRA Greenhouse to support evidence-informed practices and 

assessment-guided instructional decision-making 

“l used them [assessment results] as a weekly reminder to see those ones that are struggling. If 

you're not focused on them, you can forget, so the scores helped me see who | needed to focus 
on, who needed extra support- it helped me be pro-active.” 


“I like all of it, but definitely the activities monitoring. | liked having the scores for literacy to help 
remind me who will need my support.” 


“I liked the suggested phrasing, especially for the carpet time. It was helpful to keep me focused 
on where we’re supposed to be.” 


“I was able to give it [the Table Topper resource] to my assistant... to put us on the same page.’ 


“I looked at them [Word lists] beforehand and then when | needed help to think of words. It was 
especially helpful when we were looking for pictures to put in the classroom.” 


Impact on Practice: Impact of LRA Greenhouse on how teachers develop children’s literacy 
skills was a key tool endeavor. 


We found two central areas of impact on teacher practice: (1) teacher mindsets and (2) 
classroom practices. Teachers reported changes in their thinking and in their approach to 


helping all children develop crucial literacy skills in time for learning to read in Kindergarten. 
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Both areas demonstrate the deep level of teacher reflection that grew from having purposeful, 


doable, and clear aims to reduce children’s risk for RD. 


(1) Shifting Teacher Mindsets: 


“My biggest takeaway is how it’s [assessment, curriculum, teaching, learning] all connected. | 
knew that in the back of my mind, but using this curriculum has reminded me how it’s all 
connected and seeing it all put together really put it to the forefront.” 


“Even though we did individualizing before Greenhouse, it changed our focus on the children 
who were behind.” 


“It really made me take a look at the way kids are doing activities...Previously | didn’t think 
about a preschooler being at-risk because it’s not required [that they learn] to read.” 


“| think it [my teaching] has changed a lot. | see now that we can teach them the skills and 
behaviors needed rather than just rote memorization of letters.” 


“It has totally changed how I’m approaching things. The children now do sound when 
they see letters. Even out and about, they’re recognizing those specific [learned] letters.” 


(2) Shifting Classroom Practices—Enhanced Prevention-Based Skills: 


“It helped me remember which kids | wanted to call on [to check on learning progress]. It helped 
me to put them into groups.” 


“I would just tie my own planning into the planning, so everything was connected. A lot of times, 
my days would be based on the activities from LRA Greenhouse.” 


“It helped me start to be more specific at how | divided up the kids... be more aware that they 
would need more guidance. I’m trying to find more time to do more 1:1 instead of just as a 
group.” 

“It made my support of them [children with High Priority Needs] better, and made me feel 


confident. It gave me more specific ways to support them and the language | needed, which 
was nice. It made my support of them better, and made me feel confident.” 


Regarding changes in mindsets (quotes in 1, above), teachers regularly mentioned how 
the LRA Greenhouse was “specific”, and how this perceived specificity helped them to feel clear 


about their purpose and understanding of how their actions were connected to children’s 
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literacy learning. Teachers also shared that the structure of the tool enabled them to reflect on 
their practice differently (quotes in 2, above—both in terms of building a greater understanding 
for relations between assessment-guided intentional teaching practices and how they attended 
to children’s learning. In their willingness to implement new strategies for their classrooms, 
teachers reported changing their practice in ways that they were not explicitly trained, 
involving instructional planning, grouping, and initiating personalized supports. Anecdotally, 
teachers admitted that despite the effort they exerted to learn a new program, they were 


enthusiastic about how they could expand on its use the following school year. 


Discussion 


We found promising evidence that LRA Greenhouse tool use statistically and 
meaningfully improved Prekindergarten children’s pre-reading levels and positively impacted 
teachers’ mindsets and practices over the course of six months (nine days per month). In 
particular, through technology-managed and intentional teaching scaffolds, undergirded by 
relevant and feasible investments in online training, Oregon Prekindergarten teachers working 
in diverse settings adopted new assessment-guided approaches to literacy learning that sizably 
reduced RD risk in their classrooms. 

A number of important factors contributed to our findings—factors that we argue can 
inform future developers of Prekindergarten educational tools. First, great care was taken to 
advance our original research-based ideas to better “fit” with what the early childhood field 
stated was needed. In this light, we worked to create a bridge between evidence-informed 


practices known to prevent RD in the schools and quality early childhood practices that are 


79 


important for developing “the whole child” (in which, prevention approaches can seem in 
opposition through their focus on strengthening early identified weaknesses). We developed 
this tool in close connection with the teachers so that the LRA Greenhouse would be developed 
with teachers, for teachers. Second, our use of technology was purposeful and designed to help 
remedy support gaps and inequities in the field. We embedded a variety of strategies within 
the tool to efficiently and effectively provide Prekindergarten teachers an opportunity to learn 
critical skills and develop greater precision for teaching literacy fundamentals, administer 
assessment and use the results to provide purpose-driven supports (particularly for children 
identified with High Priority Needs), and monitor and evaluate the effectiveness of their actions 
to promote timely learning. Third, we aimed for the LRA Greenhouse to permit instructional 
flexibility within its structure. Achieving balance in all aspects of development was challenging. 
Our team regularly weighed the trade-offs of design choices to create a tool that could make 
effective research-based practices accessible and worth “trying out”, yet sustainable over time. 


There are also limitations to these findings. First, this tool worked most successfully in 


higher-quality rated Preschool classrooms with experienced teachers. Teachers in these 
classrooms had some experience with assessment, and although they had not used it to guide 
instruction, could envision the benefits of using such information to individually support 
children at-risk for RD. In most cases, teachers also had supportive leadership, assistants, and 
parents. These human resources played important roles in both the enthusiasm and 


sustainability of trying innovative approaches. Second, although pilot study classrooms were 


diverse in many ways, the racial and ethnic composition of both children and teachers was not 


representative of all of Oregon nor the broader United States. Consequently, our findings may 
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have limited generalizability to Prekindergarten sites located in urban or rural areas of Oregon 
or in other states. Future development might focus on adapting the strategies we found fruitful 
in more racially and ethnically diverse settings. Third, although the research team was trained 
to not provide “coaching” (e.g., by recommending improvements in practice or giving specific 
praise), teachers were appreciative of the partnership that formed with team members over 
the course of their participation. It is likely that this benefit to participation positively 
influenced teachers’ fairly easy up-take of the innovation and sustained motivation through 
“bumps in the road”. We note this because these results may not be found in cases without this 
“spill-over” effect. 
Conclusion 

We believe that our preliminary findings are encouraging. Prekindergarten teachers 
implemented a strategic literacy + behavior regulation curriculum that contained mostly 
unknown evidence-informed strategies for learning and that required the use of technology in 
all aspects of a 5-step assessment-guided instructional decision-making model. Although we 
strived to keep instructional preparation, activity plan understanding, and assessment 
administration relevant and efficient, LRA Greenhouse implementation did initially entail 
additional prep time beyond typical practices. Our findings from examining teacher fidelity 
indicated that they could sustain their commitment to reducing children’s risk for RD in time for 
starting Kindergarten through the use of the tool’s wrap-around and embedded supports. 

Teachers not only adopted new practices, but they deepened their knowledge of their 
children in ways that improved the quality of children’s learning and reduced their risk for RD. 


Generally speaking, sustainably adopting new evidence-informed approaches involves change 
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that is never easy, even when simple. Thus, ensuring that new approaches are successful in 
Preschool requires striking a balance between implementing the research-based strategies that 
we know work in ways that respect the diverse philosophies and cultures of learning in these 
settings. Our current and future work on the LRA Greenhouse frames striking this balance as 
fundamental to ensuring all children successfully transition Kindergarten in ways that prevent 


the development of persistent RD. 


82 


References 

Al Otaiba, S., Connor, C., Lane, H., Kosanovich, M. L., Schatschneider, C., Dyrlund, A. K., ... & 
Wright, T. L. (2008). Reading First kindergarten classroom instruction and students’ 
growth in phonological awareness and letter naming—decoding fluency. Journal of 
School Psychology, 46(3), 281-314. 

Al Otaiba, S., Hosp, J. L., Smartt, S., & Dole, J. A. (2008). The challenging role of a reading coach, 
a cautionary tale. Journal of Educational and Psychological Consultation, 18(2), 124-155. 

Algozzine, B., Ysseldyke, J. E., & Shinn, M. (1982). Identifying children with learning disabilities: 
When is a discrepancy severe? Journal of School Psychology, 20(4), 299-305. 

Alloway, T. P., & Alloway, R. G. (2010). Investigating the predictive roles of working memory and 
IQ in academic attainment. Journal of Experimental Child Psychology, 106, 20-29. 

Alloway, T. P., Gathercole, S. E., Kirkwood, H., & Elliott, J. (2009). The cognitive and behavioral 
characteristics of children with low working memory. Child Development, 80, 2, 606- 

621. 

Anthony, J. L., & Lonigan, C. J. (2004). The nature of phonological awareness: Converging 
evidence from studies of preschool and early grade school children. Journal of 
Educational Psychology, 96, 1, 43-55. 

Anthony, J. L., Lonigan, C. J., Driscoll, K., Phillips, B. M.., & Burgess, S. R. (2003). Phonological 
sensitivity: A quasi-parallel progression of word structure units and cognitive operations. 
Reading Research Quarterly, 38, 4, 470-487. 

Archer, A. L., & Hughes, C. A. (2011). Explicit instruction: Effective and efficient instruction. New 


York, NY: Guilford Publications. 


83 


Atkins-Burnett, S. (2007). Measuring children’s progress from preschool through third grade 
(No. 2d7310d35a3e4a129792dde6d1f5107b). Princeton, NJ: Mathematica Policy 
Research. 

Baddeley, A. D., & Hitch, G. (1974). Working memory. Psychology of Learning and Motivation, 8, 
47-89. 

Bailet, L. L., Repper, K. K., Piasta, S. B., & Murphy, S. P. (2009). Emergent literacy intervention 
for prekindergarteners at risk for reading failure. Journal of Learning Disabilities, 42(4), 
336-355. 

Bassok, D., Latham, S., & Rorem, A. (2016). Is kindergarten the new first grade? AERA Open, 
2(1), 1-31. DOI: 10.1177/2332858415616358. 

Bast, J., & Reitsma, P. (1998). Analyzing the development of individual differences in terms of 
Matthew effects in reading: Results from a Dutch longitudinal study. Developmental 
Psychology, 34(6), 1373-1379. 

Bayliss, D. M., Jarrold, C., Baddeley, A. D., & Gunn, D. M. (2005). The relationship between 
short-term memory and working memory: Complex span made simple? Memory, 13(3- 
4), 414-421. 

Belke, E. (2008). Effects of working memory load on lexical-semantic encoding in language 
production. Psychonomic Bulletin &Review, 15(2), 357-363. 

Berger, A., Jones, L., Rothbart, M. K., & Posner, M. I. (2000). Computerized games to study the 
development of attention in childhood. Behavior Research Methods, Instruments, & 


Computers, 32(2), 297-303. 


84 


Bergsmann, E. M., Luftenegger, M., Jostl, G., Schober, B., & Spiel, C. (2013). The role of 
classroom structure in fostering students’ school functioning: A comprehensive and 
application-oriented approach. Learning and Individual Differences, 26, 131-138. 

Bierman, K. L., Torres, M. M., Domitrovich, C. E., Welsh, J. A., & Gest, S. D. (2008). Behavioral 
and cognitive readiness for children attending Head Start. Social Development, 18, 2, 
305- 323. 

Bilmes, J. (2004). Beyond behavior management: The six life skills children need to thrive in 
today's world. Saint Paul, MN: Redleaf Press. 

Blair, C., & Diamond, A. (2008). Biological processes in prevention and intervention: The 
promotion of self-regulation as a means of preventing school failure. Development and 
Psychopathology, 20, 899-911. 

Blair, C., & Raver, C. C. (2015). School readiness and self-regulation: A developmental 
psychobiological approach. Annual Review of Psychology, 66, 711-731. 

Blase, K., Van Dyke, M., & Fixsen, D. (2013). Stages of implementation analysis: Where are we? 
Chapel Hill, NC: National Implementation Research Network. 

Boscardin, C. K., Muthén, B., Francis, D.J., & Baker, E. L. (2008). Early identification of reading 
difficulties using heterogeneous developmental trajectories. Journal of Educational 
Psychology, 100, 1, 192-208. 

Bronson, M. B., Tivnan, T., & Seppanen, P. S. (1995). Relations between teacher and classroom 
activity variables and the classroom behaviors of prekindergarten children in Chapter 1 


funded programs. Journal of Applied Developmental Psychology, 16(2), 253-282. 


85 


Buysse, V., & Peisner-Feinberg, E. (2010). Recognition & response: Response to intervention for 
prek. Young Exceptional Children, 13(4), 2-13. 

Cabell, S. Q., Justice, L. M., Konold, T. R., & McGinty, A. S. (2011). Profiles of emergent literacy 
skills among preschool children who are at risk for academic difficulties. Early Childhood 
Research Quarterly, 26(1), 1-14. 

Carson, K., Gillon, G., & Boustead, T. (2011). Computer-administrated versus paper-based 
assessment of school-entry phonological awareness ability. Asia Pacific Journal of 
Speech, Language and Hearing, 14(2), 85-101. 

Chard, D. J., Stoolmiller, M., Harn, B. A., Wanzek, J., Vaughn, S., Linan-Thompson, S., & 
Kame'enui, E. J. (2008). Predicting reading success in a multilevel schoolwide reading 
model: A retrospective analysis. Journal of Learning Disabilities, 41(2), 174-188. 

Clark, R. C., & Mayer, R. E. (2003). E-Learning: promise and pitfalls. E-Learning and the Science 
of Instruction, 11-31. 

Coie, J. D., & Krehbiel, G. (1984). Effects of academic tutoring on the social status of low- 
achieving, socially rejected children. Child Development, 55, 1465-1478. 

Compton, D. L., Fuchs, L. S., Fuchs, D., Lambert, W., & Hamlett, C. (2012). The cognitive and 
academic profiles of reading and mathematics learning disabilities. Journal of Learning 
Disabilities, 45(1), 79-95. 

Connolly, J. A., & Doyle, A. B. (1984). Relation of social fantasy play to social competence in 
preschoolers. Developmental Psychology, 20(5), 797- 806. 

Connor, C. M., Morrison, F. J., & Slominski, L. (2006). Preschool instruction and children's 


emergent literacy growth. Journal of Educational Psychology, 98(4), 665-689. 


Conway, A. R., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). 
Working memory span tasks: A methodological review and user’s guide. Psychonomic 
Bulletin & Review, 12 (5), 769-786. 

Courtney, S. M. (2004). Attention and cognitive control as emergent properties of information 
representation in working memory. Cognitive, Affective, & Behavioral Neuroscience, 
4(4), 501-516. 

Cowan, N. (2010). The magical mystery four: How is working memory capacity limited, and 
why? Current Directions in Psychological Science, 19(1), 51-57. 

Dally, K. (2006). The influence of phonological processing and inattentive behavior on reading 
acquisition. Journal of Educational Psychology, 98, 2, 420-437. 

Davis, F. D. (1993). User acceptance of information technology: system characteristics, user 
perceptions and behavioral impacts. International Journal of Man-machine Studies, 
38(3), 475-487. 

Deans for Impact (2019). The Science of Early Learning. Austin, TX: Deans for Impact. 

Division for Early Childhood (2019). Multi-tiered system of support framework in early 
childhood: Description and implications (Revised position statement draft). Author. 
Retrieved from: https://www.dec-sped.org/position-statements-development. 

de Jong, P. F. (1998). Working memory deficits of reading disabled children. Journal of 
Experimental Child Psychology, 70, 75-96. 

de Jong, P. F., & van der Leij, A. (2003). Developmental changes in the manifestation of a 
phonological deficit in dyslexic children learning to read a regular orthography. Journal 


of Educational Psychology, 95(1), 22. 


86 


87 


Dempster, F. N. (1988). The spacing effect: A case study in the failure to apply the results of 
psychological research. American Psychologist, 43(8), 627- 634. 

Denham, S. A., Blair, K. A., Demulder, E., Levitas, J., Sawyer, K., Auerbach-Maijor, S., & Queenan, 
P. (2003). Preschool emotional competence: Pathway to social competence? Child 
Development, 74, 1, 238-256. 

Diamond, K. E., Justice, L. M., Siegler, R. S., & Snyder, P. A. (2013). Synthesis of IES research on 
early intervention and early childhood education. NCSER 2013-3001. Washington, DC: 
National Center for Special Education Research. 

Drouin, M., Horner, S. L., & Sondergeld, T. A. (2012). Alphabet knowledge in preschool: A Rasch 
model analysis. Early Childhood Research Quarterly, 27(3), 543-554. 

Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., et al. 
(2007). School readiness and later achievement. Developmental Psychology, 43, 1428- 
1446. 

Eisenberg, N., Cumberland, A., Spinrad, T. L., Fabes, R. A., Shepard, S. A. Reiser, M., ... & Guthrie, 
I. K. (2001). The relations of regulation and emotionality to children’s externalizing and 
internalizing problem behavior. Child Development, 72, 4, 1112-1134. 

Engle, R. W. (2010). Role of working-memory capacity in cognitive control. Current 
Anthropology, 51(S1), $17-S26. 

Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, 
confidence, beliefs, and culture intersect. Journal of Research on Technology in 


Education, 42(3), 255-284. 


88 


Feldon, D. F. (2007). Cognitive load and classroom teaching: The double-edged sword of 
automaticity. Educational Psychologist, 42(3), 123-137. 

Fletcher, J. M., Lyon, G. R., & Fuchs, L. S., & Barnes, M.A. (2007). Learning disabilities: From 
identification to intervention. New York, NY: Guilford press. 

Flook, L., Goldberg, S. B., Pinger, L., & Davidson, R. J. (2015). Promoting prosocial behavior and 
self-regulatory skills in preschool children through a mindfulness-based kindness 
curriculum. Developmental Psychology, 51(1), 44- 51. 

Friedman-Krauss, A. H., Barnett, W. S., Garver, K. A., Hodges, K. S., Weisenfeld, G. G., & 
DiCrecchio, N. (2019). The State of Preschool 2018: State Preschool Yearbook. National 
Institute for Early Education Research. 

Fuchs, D., & Fuchs, L. S. (2006). Introduction to response to intervention: What, why, and how 
valid is it? Reading Research Quarterly, 41(1), 93-99. 

Gaggi, O., Galiazzo, G., Palazzi, C., Facoetti, A., & Franceschini, S. (2012, July). A serious game for 
predicting the risk of developmental dyslexia in pre-readers children. 21st International 
Conference on Computer Communications and Networks (ICCCN), 1-5. 

Gaias, L. M., Abry, T., Swanson, J., & Fabes, R. A. (2016). Considering child effortful control in 
the context of teacher effortful control: Implications for kindergarten success. Learning 
and Individual Differences, 45, 199-207. 

Garon, N., Bryson, S. E., & Smith, I. M. (2008). Executive function in preschoolers: a review using 


an integrative framework. Psychological Bulletin, 134(1), 31-60. 


89 


Gathercole, S. E., & Pickering, S. J. (2000). Working memory deficits in children with low 
achievements in the national curriculum at 7 years of age. British Journal of Educational 
Psychology, 70(2), 177-194. 

Gathercole, S. E., Alloway, T. P., Kirkwood, H. J., Elliott, J. G., Holmes, J., & Hilton, K. A. (2008). 
Attentional and executive function behaviors in children with poor working memory. 
Learning and Individual Differences, 18, 214-223. 

Gathercole, S. E., Pickering, S. J., Ambridge, B., & Wearing, H. (2004). The structure of working 
memory from 4 to 15 years of age. Developmental Psychology, 40, 2, 177-190. 

Gavens, N., & Barrouillet, P. (2004). Delays of retention, processing efficiency, and attentional 
resources in working memory span development. Journal of Memory and Language, 
51(4), 644-657. 

Goble, P., Eggum-Wilkens, N. D., Bryce, C. I., Foster, S. A., Hanish, L. D., Martin, C. L., & Fabes, R. 
A. (2017). The transition from preschool to first grade: A transactional model of 
development. Journal of Applied Developmental Psychology, 49, 55-67. 

Greenwood, C. R., Bradfield, T., Kaminski, R., Linas, M., Carta, J. J., & Nylander, D. (2011). The 
response to intervention (RTI) approach in early childhood. Focus on Exceptional 
Children, 43(9), 1-22. 

Greenwood, C. R., Carta, J. J., Atwater, J., Goldstein, H., Kaminski, R., & McConnell, S. (2013). Is 
a Response to Intervention (RTI) approach to preschool language and early literacy 


instruction needed? Topics in Early Childhood, 33 (1), 48-64. 


90 


Groot, A. S., De Sonneville, L. M., Stins, J. F., & Boomsma, D. |. (2004). Familial influences on 
sustained attention and inhibition in preschoolers. Journal of Child Psychology and 
Psychiatry, 45(2), 306-314. 

Halberstadt, A. G., Denham, S. A., & Dunsmore, J. C. (2001). Affective Social Competence. Social 
Development, 10(1), 79-119. 

Hamilton, L., Halverson, R., Jackson, S. S., Mandinach, E., Supovitz, J. A., & Wayman, J. (2009). 
Using student achievement data to support instructional decision making (NCEE 209- 
4067). Washington, DC: National Center for Education Evaluation and Regional 
Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved 
from: https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/dddm 092909. pdf. 

Hamre, B. K., & Pianta, R. C. (2001). Early teacher-child relationships and the trajectory of 
children’s school outcomes through eighth grade. Child Development, 72, 2, 625-638. 

Hindson, B., Byrne, B., Fielding-Barnsley, R., Newman, C., Hine, D. W., & Shankweiler, D. (2005). 
Assessment and early instruction of preschool children at risk for reading disability. 
Journal of Educational Psychology, 97(4), 687- 704. 

Hinkle, H. M., Irvin. P. S., Pilger, M., & Saez. L. (2018, October). Data-based decision-making: 
Exploring preschool teachers’ role in reading disability prevention. Poster presented at 
the 40th International Conference on Learning Disabilities (CLD), Portland, OR. 

Hoover, J. J. (2011). Making informed instructional adjustments in RT| models: Essentials for 


practitioners. Intervention in School and Clinic, 47(2), 82-90. 


91 


Howse, R. B., Calkins, S. D., Anastopoulos, A.D., Keane, S. P., & Shelton, T. L. (2003). Regulatory 
contributors to children’s kindergarten achievement. Early Education and Development, 
14, 101- 119. 

Irvin, P. S., Pilger, M., Saez, L., & Alonzo, J. (2016). Innovation need survey: Implementing a 
technology tool to improve early data-based decisions to address and prevent learning 
disabilities (Technical Report 1602). Eugene, OR: Behavioral Research and Teaching. 

Johnson, E. S., Humphrey, M., Mellard, D. F., Woods, K., & Swanson, H. L. (2010). Cognitive 
processing deficits and students with specific learning disabilities: A selective meta- 
analysis of the literature. Learning Disability Quarterly, 33(1), 3-18. 

Justice, L. M., Chow, S., Capellini, C., Flanigan, K., & Colton, S. (2003). Emergent literacy 
intervention for vulnerable preschoolers: Relative effects of two approaches. American 
Journal of Speech-Language Pathology, 12, 320-332. 

Justice, L. M., & Kaderavek, J. N. (2004). Embedded-explicit emergent literacy intervention I. 
Language, Speech, and Hearing Services in Schools,35, 3, 201-211. 

Justice, L. M., Mashburn, A. J., Hamre, B. K., & Pianta, R. C. (2008). Quality of language and 
literacy instruction in preschool classrooms serving at-risk pupils. Early Childhood 
Research Quarterly, 23(1), 51-68. 

Justice, L. M., Pence, K., Bowles, R. B., & Wiggins, A. (2006). An investigation of four hypotheses 
concerning the order by which 4-year-old children learn the alphabet letters. Early 
Childhood Research Quarterly, 21(3), 374-389. 

Kaderavek, J. N., & Justice, L. M. (2004). Embedded-Explicit Emergent Literacy Intervention Il. 


Language, Speech, and Hearing Services in Schools, 35, 3, 212-228. 


92 


Kavale, K. A., & Forness, S. R. (2000). What definitions of learning disability say and don't say: A 
critical analysis. Journal of Learning Disabilities, 33(3), 239-256. 

Kim, Y. - S., Petscher, Y., Foorman, B. R., & Zhou, C. (2010). The contributions of phonological 
awareness and letter-name knowledge to letter-sound acquisition: A cross-classified 
multilevel model approach. Journal of Educational Psychology, 102, 2, 313-326. 

Kofler, M. J., Spiegel, J. A., Soto, E. F., Irwin, L. N., Wells, E. L., & Austin, K. E. (2018). Do working 
memory deficits underlie reading problems in attention-deficit/hyperactivity disorder 
(ADHD)? Journal of Abnormal Child Psychology, 47(3), 433-446. 

La Paro, K. M., & Pianta, R. C. (2000). Predicting children's competence in the early school years: 
A meta-analytic review. Review of Educational Research, 70(4), 443-484. 

Lin, H., Lawrence, F. R., & Gorrell, J. (2003). Kindergarten teachers' views of children's readiness 
for school. Early Childhood Research Quarterly, 18, 225-237. 

Lonigan, C. J., Burgess, S. R., & Anthony, J. L. (2000). Development of emergent literacy and 
early reading skills in preschool children: Evidence from a latent-variable longitudinal 
study. Developmental Psychology, 36, 5, 596-613. 

Lundberg, I., Frost, J., & Petersen, O. P. (1988). Effects of an extensive program for stimulating 
phonological awareness in preschool children. Reading Research Quarterly, 263-284. 

Lyon, G. R. (1994). Frames of reference for the assessment of learning disabilities: New views on 
measurement issues. Baltimore, MD: Paul H. Brookes Publishing. 

Mandinach, E. B., & Jimerson, J. B. (2016). Teachers learning how to use data: A synthesis of the 


issues and what is known. Teaching and Teacher Education, 60, 452-457. 


93 


McBride-Chang, C. (1999). The ABCs of the ABCs: The development of letter-name and letter- 
sound knowledge. Merrill-Palmer Quarterly, 45, 2, 285-308. 

McClelland, M. M., Acock, A. C., & Morrison, F. J. (2006). The impact of kindergarten learning- 
related skills on academic trajectories at the end of elementary school. Early Childhood 
Research Quarterly, 21(4), 471-490. 

McClelland, M. M., & Cameron, C. E. (2012). Self-regulation in early childhood: Improving 
conceptual clarity and developing ecologically valid measures. Child Development 
Perspectives, 6(2), 136-142. 

McClelland, M. M., & Morrison, F. J. (2003). The emergence of learning-related social skills in 
preschool children. Early Childhood Research Quarterly, 18(2), 206-224. 

McClelland, M. M., Morrison, F. J., & Holmes, D. L. (2000). Children at-risk for early academic 
problems: The role of learning-related social skills. Early Childhood Research Quarterly, 
15(3), 307-329. 

McDonald, A., Haslam, C., Yates, P., Gurr, B., Leeder, G., & Sayers, A. (2011). Google calendar: A 
new memory aid to compensate for prospective memory deficits following acquired 
brain injury. Neuropsychological Rehabilitation, 21(6), 784-807. 

McLoughlin, C., & Marshall, L. (2000, February). Scaffolding: A model for learner support in an 
online teaching environment. In Flexible Futures in Tertiary Teaching: Proceedings of the 
9th Annual Teaching Learning Forum (Vol. 2, No. 4). Retrieved from: http://www.c3l.uni- 


oldenburg.de/cde/support/readings/loughlin2.htm 


94 


Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). 
The unity and diversity of executive functions and their contributions to complex 
“frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49-100. 

Morgan, P.L., Farkas, G., Tufis, P. A., & Sperling, R. A. (2008). Are reading and behavior 
problems risk factors for each other? Journal of Learning Disabilities, 41(5), 417-436. 

Morgan, P.L., Farkas, G., Wang, Y., Hillemeier, M. M., Oh, Y., & Maczuga, S. (2019). Executive 
function deficits in kindergarten predict repeated academic difficulties across 
elementary school. Early Childhood Research Quarterly, 46, 20-32. 

Morrison, J. W., Storey, P., & Zhang, C. (2011). Accessible family involvement in early childhood 
programs. Dimensions of Early Childhood, 39(3), 21-26. 

National Association for the Education of Young Children (2020, March). Our mission and 
strategic direction. https://www.naeyc.org/about-us/people/mission-and-strategic- 
direction. 

National Early Literacy Panel. (2009). Developing early literacy: Report of the National Early 
Literacy Panel. Washington, DC: National Institute for Literacy. 

Nguyen, T., & Duncan, G. J. (2019). Kindergarten components of executive function and third 
grade achievement: A national study. Early Childhood Research Quarterly, 46, 49-61. 

Olson, J. L., & Mealor, D. J. (1981). Learning disabilities identification: Do researchers have the 
answer? Learning Disability Quarterly, 4(4), 389-392. 

Oregon Department of Education (2016). Oregon’s early learning and kindergarten guidelines. 


Author. 


95 


Ozernov-Palchik, O., Norton, E. S., Sideridis, G., Beach, S. D., Wolf, M., Gabrieli, J. D., & Gaab, N. 
(2017). Longitudinal stability of pre-reading skill profiles of kindergarten children: 
implications for early screening and theories of reading. Developmental Science, 20(5), 
1-18. 

Phillips, B. M., Clancy-Menchetti, J., & Lonigan, C. J. (2008). Successful phonological awareness 
instruction with preschool children. Topics in Early Childhood Special Education, 28, 1, 3- 
17. 

Phillips, B. M., Piasta, S. B., Anthony, J. L., Lonigan, C.J., & Francis, D. J. (2012). IRTs of the ABCs: 
Children's letter name acquisition. Journal of School Psychology, 50(4), 461-481. 

Pianta, R. C., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F. J. (2008). Classroom effects on 
children’s achievement trajectories in elementary school. American Educational 
Research Journal, 45, 2, 364-397. 

Pretti-Frontczak, K., Kowalski, K., & Brown, R. D. (2002). Preschool teachers' use of assessments 
and curricula: A statewide examination. Exceptional Children, 69(1), 109-123. 

Raver, C. C., Jones, S. M., Li-Grining, C., Zhai, F., Bub, K., & Pressler, E. (2011). CSRP’s impact on 
low-income preschoolers’ preacademic skills: self-regulation as a mediating mechanism. 
Child Development, 82(1), 362-378. 

Ratsavong, L., Myrtil, M., Gugiu, M., & Schachter, R. (Winter 2016). Bridging the Gap between 
Data and Instruction to Promote School Readiness. Columbus, OH: Crane Center for 


Early Childhood Research and Policy, The Ohio State University. 


96 


Reilly, S. E., & Downer, J. T. (2019). Roles of executive functioning and language in developing 
low-income preschoolers’ behavior and emotion regulation. Early Childhood Research 
Quarterly, 49, 229-240. 

Rimm-Kaufman, S. E., & Pianta, R. C. (2000). An ecological perspective on the transition to 
kindergarten: A theoretical framework to guide empirical research. Journal of Applied 
Developmental Psychology, 21(5), 491-511. 

Rimm-Kaufman, S. E., Pianta, R. C., & Cox, M. J. (2000). Teachers’ judgments of problems in the 
transition to kindergarten. Early Childhood Research Quarterly, 15(2), 147-166. 

Roehrig, A. D., Duggar, S. W., Moats, L., Glover, M., & Mincey, B. (2008). When teachers work to 
use progress monitoring data to inform literacy instruction: Identifying potential 
supports and challenges. Remedial and Special Education, 29(6), 364-382. 

Rogers, M., Hwang, H., Toplak, M., Weiss, M., & Tannock, R. (2011). Inattention, working 
memory, and academic achievement in adolescents referred for attention 
deficit/hyperactivity disorder (ADHD). Child Neuropsychology, 17(5), 444-458. 

Saez, L., Folsom, J. S., Al Otaiba, S., & Schatschneider, C. (2012). Relations among student 
attention behaviors, teacher practices, and beginning word reading skill. Journal of 
Learning Disabilities, 45 (5), 418-432. 

Saez, L., Nese, J. F., Alonzo, J., & Tindal, G. (2016). Individual differences in kindergarten 
through grade 2 fluency relations. Learning and Individual Differences, 49, 100-109. 

Saez, L., & Pilger, M. (November, 2016). The Learning Receptiveness Assessment: A tablet- 


based tool for bridging kindergarten and preschool early screenings to prevent 


97 


learning difficulties. Poster presented at the annual meeting of the National 
Association for the Education of Young Children (NAEYC), Los Angeles, CA. 

Schatschneider, C., Fletcher, J. M., Francis, D. J., Carlson, C. D., & Foorman, B. R. (2004). 
Kindergarten prediction of reading skills: A longitudinal comparative analysis. Journal of 
Educational Psychology, 96, 2, 265-282. 

Schatschneider, C., Francis, D. J., Foorman, B. R., Fletcher, J. M., & Mehta, P. (1999). The 
dimensionality of phonological awareness: An application of item response theory. 
Journal of Educational Psychology, 91,3, 439-449. 

Schmeichel, B. J., Volokhov, R. N., & Demaree, H. A. (2008). Working memory capacity and the 
self-regulation of emotional expression and experience. Journal of Personality and Social 
Psychology, 95, 6, 1526-1540. 

Schonert-Reichl, K. A., Oberle, E., Lawlor, M. S., Abbott, D., Thomson, K., Oberlander, T. F., & 
Diamond, A. (2015). Enhancing cognitive and social-emotional development through a 
simple-to-administer mindfulness-based school program for elementary school children: 
A randomized controlled trial. Developmental psychology, 51(1), 52- 66. 

Schwebel, D. C., Rosen, C. S., & Singer, J. L. (1999). Preschoolers' pretend play and theory of 
mind: The role of jointly constructed pretence. British Journal of Developmental 
Psychology, 17(3), 333-348. 

Siegel, L. S., & Ryan, E. B. (1989). The development of working memory in normally achieving 


and subtypes of learning disabled children. Child Development, 60, 973-980. 


98 


Skibbe, L. E., Montroy, J. J., Bowles, R. P., & Morrison, F. J. (2019). Self-regulation and the 
development of literacy and language achievement from preschool through second 
grade. Early childhood research quarterly, 46, 240-251. 

Simmons, D. C., Coyne, M. D., Kwok, O., McDonagh, S., Harn, B. A., & Kame’enui, E. J. (2008). 
Indexing Response to Intervention: A longitudinal study of reading risk from 
kindergarten through third grade. Journal of Learning Disabilities, 41, 2, 158-173. 

Simmons, D. C., Kame'enui, E. J., Harn, B., Coyne, M. D., Stoolmiller, M., Edwards Santoro, L., ... 
& Kaufman, N. K. (2007). Attributes of effective and efficient kindergarten reading 
intervention: An examination of instructional time and design specificity. Journal of 
Learning Disabilities, 40(4), 331-347. 

Smith, B., Hurth, J., Pletcher, L., Shaw, E., Whaley, K., Peters, M., & Dunlap, G. (2014). A guide to 
the implementation process: Stages, steps, and activities. The National Implementation 
Research Network (NIRN) University of North California Chapel Hill. Available at: 
http://ectacenter. org/~ pdfs/implementprocess/implementprocess-stagesandsteps. pdf. 

Smith, J. L. M., Saez, L., & Doabler, C. T. (2016). Using explicit and systematic instruction to 
support working memory. TEACHING Exceptional Children, 48(6), 275-281. 

Stecker, P. M., Lembke, E. S., & Foegen, A. (2008). Using progress-monitoring data to improve 
instructional decision making. Preventing School Failure: Alternative Education for 
Children and Youth, 52(2), 48-58. 

Storch, S. A., & Whitehurst, G. J. (2002). Oral language and code-related precursors to reading: 
evidence from a longitudinal structural model. Developmental psychology, 38(6), 934- 


947. 


99 


Sugai, G., & Horner, R. H. (2009). Responsiveness-to-intervention and school-wide positive 
behavior supports: Integration of multi-tiered system approaches. Exceptionality, 17(4), 
223-237. 

Swanson, H. L. (1991). Operational definitions and learning disabilities: An overview. Learning 
Disability Quarterly, 14(4), 242-254. 

Swanson, H. L., & Saez, L. (2003). Memory difficulties in children and adults with learning 
disabilities. Handbook of Learning Disabilities, 182-198. 

Swanson, H. L., Howard, C. B., & Saez, L. (2006). Do different components of working memory 
underlie different subgroups of reading disabilities? Journal of Learning Disabilities, 
39(3), 252-269. 

Snyder, T.D., de Brey, C., and Dillow, S.A. (2019). Digest of education statistics 2017 (NCES 2018- 
070, Table 202.30). National Center for Education Statistics, Institute of Education 
Sciences, U.S. Department of Education. Washington, DC. 

Taylor, H. G., Anselmo, M., Foreman, A. L., Schatschneider, C., & Angelopoulos, J. (2000). Utility 
of kindergarten teacher judgments in identifying early learning problems. Journal of 
Learning Disabilities, 33(2), 200-210. 

Torgesen, J. K. (1998). Catch them before they fall: Identification and assessment to prevent 
reading failure in young children. American Educator, 22, 32-39. 

Torgesen, J. K., Wagner, R. K., Rashotte, C. A., Rose, E., Lindamood, P., Conway, T., & Garvan, C. 
(1999). Preventing reading failure in young children with phonological processing 
disabilities: Group and individual responses to instruction. Journal of Educational 


Psychology, 91(4), 579. 


100 


Treiman, R., & Zukowski, A. (1996). Children’s sensitivity to syllables, onsets, rimes, and 
phonemes. Journal of Experimental Child Psychology, 61, 193-215. 

Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent? Journal of 
Memory and Language, 28(2), 127-154. 

VanDerHeyden, A. M., Snyder, P. A., Broussard, C., & Ramsdell, K. (2008). Measuring response 
to early literacy intervention with preschoolers at risk. Topics in Early Childhood Special 
Education, 27(4), 232-249. 

Van den Hurk, H. T. G., Houtveen, A. A. M., & Van de Grift, W. J. C. M. (2016). Fostering 
effective teaching behavior through the use of data-feedback. Teaching and Teacher 
Education, 60, 444-451. 

Wagner, R. K., Torgesen, J. K., Rashotte, C. A., Hecht, S. A., Barker, T. A., Burgess, S. R., ... & 
Garon, T. (1997). Changing relations between phonological processing abilities and 
word-level reading as children develop from beginning to skilled readers: a 5-year 
longitudinal study. Developmental Psychology, 33, 3, 468. 

Welsh, J. A., Nix, R. L., Blair, C., Bierman, K. L., & Nelson, K. E. (2010). The development of 
cognitive skills and gains in academic school readiness for children from low-income 
families. Journal of Educational Psychology, 102(1), 43-53. 

Wesley, P. W., & Buysse, V. (2003). Making meaning of school readiness in schools and 
communities. Early Childhood Research Quarterly, 18(3), 351-375. 

Whitebook, M., & Sakai, L. (2003). Turnover begets turnover: An examination of job and 
occupational instability among child care center staff. Early Childhood Research 


Quarterly, 18(3), 273-293. 


101 


White-Schwoch, T., Carr, K. W., Thompson, E. C., Anderson, S., Nicol, T., Bradlow, A. R., ... & 
Kraus, N. (2015). Auditory processing in noise: A preschool biomarker for literacy. PLOS 
Biology, 13(7), 1-17. doi: 10.1371/journal.pbio.1002196. 

Williford, A. P., Whittaker, J. E., Vitiello, V. E., & Downer, J. T. (2013). Children’s engagement 
within the preschool classroom and their development of self-regulation. Early 
Educational Development, 24 (2), 162-187. 

Zelazo, P. D., & Lyons, K. E. (2012). The potential benefits of mindfulness training in early 
childhood: A developmental social cognitive neuroscience perspective. Child 


Development Perspectives, 6(2), 154-160. 


102 


Appendix 


Technical Report Results 


Table 1. Pilot Study Mean Performance for LRA Greenhouse Implementing and 


Non-Implementing Comparison Classrooms 


Total 4\implementing Classes 4 Non-Implementing Classes 
Measure : 
Possible (n = 33) (n = 37) 
(F) Pre-reading 10.79 (4.01) 10.08 (4.25) 
0-24 
(S) Pre-reading 14.97** (4.57) 11.08 (4.83) 
(F) Working Memory 0.88 (.89) 1.00 (1.00) 
0-5 
(S) Working Memory 1.58* (.97) 1.08 (.83) 
(F) Task Engaged Behavior 44.06 (13.70) 47.48 (7.38) 
0-75 
(S) Task Engaged Behavior 51.21 (16.48) 49.41 (9.32) 
(F) Prosocial Behavior 54.47* (9.96) 50.00 (5.66) 
0-75 
(S) Prosocial Behavior 56.70 (11.36) 53.46 (5.74) 


Note. ** p = .001, * p< .05. All other comparisons were non-significant. 
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Table 2. Pilot Study Mean Performance for Four LRA Greenhouse 


Partial Implementing Classrooms (n = 24) 


Measure Total Possible Score Range Mean Std. Deviation 

(W) Pre-reading 6-21 11.04 3.29 
0-24 

(S) Pre-reading 8-20 12.58* 3.51 

(W) Working Memory 0-2 0.75 0.79 
0-5 

(S) Working Memory 0-3 1.04 0.75 

(W) Task Engaged Behavior 28-62 49.71 9.54 

(S) Task Engaged Behavior 31-64 50.33 8.84 
0-75 

(W) Prosocial Behavior 35-74 52.13 8.78 

(S) Prosocial Behavior 37-72 54.33 8.46 


Note. * p = .01. All other comparisons were non-significant. 
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Table 3. Reliability for Pre-reading and BRS Components of the LRA by Season 


Pre-reading (Fall): a = .79 


BRS Total (Fall): a = .97 BRS TE(Fall): a = .94 BRS PRS (Fall): a = .95 


Pre-reading (Winter): a = .73 


BRS Total (Winter): a = .94 BRS TE (Winter): a = .94 BRS PRS (Winter): a = .92 


Pre-reading (Spring): a = .76 


BRS Total (Spring): a = .95 BRS TE (Spring): a = .93 BRS PRS (Spring): a = .93 


Note. Due to the nature of the WM task design, in which all items were not administered, alpha could 
not be calculated. 


BRS = Behavioral Rating Scale; TE = Task Engaged component of the BRS; PRS = Prosocial Engaged 
component of the BRS. 
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Table 4. Pearson Correlations of Prekindergarten LRA Measures Over Time 


Measure W_LIT S_LIT F.WM W_WM S_WM F_BRS W_BRS S_BRS F_BRS.TE W_BRS.TE S_BRS.TE F_BRS.PE W_BRS.PE S_BRS.PE 
F_LIT 77 73° ©6556 35 40 68 «.70 8.71 67 63 69" 59" 59" 59" 
W_LIT 1.00 .80 46 AL 18 1 #50 49° 39° 40° 45° 38° 49" 44° 
S_LIT ae 1.00 .48° 31 40 40 4.62. #852 38° 56. 52. 36 52° 42° 
F_WM = de 1.00 17 28 52 4.50 50. 43° 39° 39° 57. 52. A 
W_WM - “ 2 1.00 18 05 32 25 9 35 31 -.05 ag 12 
S_ WM = = 2 ae 1.00 .42 53 47” 39° 49" 41° 39° 43° 476 
F_BRS Fe x ~ Bs Ee 1.00 .80 83 = ss i x = i 
W BRS a2 = = a 2 es 1.00 .86 - es Ee e - = 
S_BRS x = = ss a = 2 = = a = = = = 
F_BRS.TE = ss 2 = i = Md = 1.00 74°" 81" 17" 55. 63 
W_BRS.TE Fe & = bs = ze 2 = ss 1.00 81 52. 52. 53. 
S_BRS.TE = se = Bs = = Pe = - = 1.00 57. 56. 69 
F_BRS.PE = = = = es = ES & a & 2 1.00 81 86 
W_BRS.PE ie Ee = ie us " 2 = = = a E 1.00 89° 


Note. Pearson correlations were obtained to examine the consistency of construct relations over time (with n = 29 children from 4 
classrooms implementing the tool across six months of the school year). Bolded correlations are significant (p < 0.05). 

*** = 9 < 001; ** =p <.01; * =p <.05; F = Fall; W = Winter; S = Spring; LIT = Pre-reading Literacy; WM = Working Memory; BRS = 
Behavior Rating Scale; TE = Task Engaged regulation (of the BRS); PE = Prosocial Engaged regulation (of the BRS). 


Participating teachers’ perceptions of tool suitability, reported via individual interviews and online surveys are organized below 
based on: Feasibility, Accessibility, Usefulness, Fit, Facilitation, and Impact on Practice. 


Feasibility — Ease and efficiency of LRA Greenhouse implementation in preschool classrooms 
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Table 5. LRA Greenhouse —Feasibility Across 39 occasions 


Survey Item n teachers M Not at all Occasionally | Somewhat Mostly Definitely 
How easy was it to implement today’s 
activities (in terms of having/finding what 6 4.67 - 1 3 4 31 
you needed)? 
How easy was it for children to grasp the P ee , 7 oA 
big ideas from today’s activities? ; ; : 
Did you encounter any hiccups that led 
you to feel that today’s implementation 6 1.44* 29 3 7 - J 
was less than ideal? 
How likely would you be to recommend 

6 4.85 - - 1 4 34 


the activities in this unit to a friend? 


Note. Reverse coded, where a lower value denotes a more positive quality (fewer “hiccups”). DA = Developmentally Appropriate. 


Accessibility — Simplicity and straightforwardness of access to and use of LRA Greenhouse components 
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Table 6a. LRA Behavior Rating Scale (BRS) — Accessibility 


screening assessments two or three times per year. 


Survey Item n M No Yes 
Creating student list was an easy process. 6 1.00 - 6 
Tablet log in/out was easy. 7 1.00 - 7 
Finding the BRS and getting into it was easy. 7 1.00 - 7 
Screen responding was easy (and sensitive to touch). 7 0.86 1 6 
BRS items on the screen were easy to see. 7 1.00 - 7 
Task instructions were clear. 7 1.00 - 7 
Navigating through the task (i.e., knowing what/where to touch for responding) was simple. 7 1.00 - 7 
| completed the task efficiently. 7 1.00 - 7 
Table 6b. LRA Greenhouse — Accessibility 

Strongly : Strongly 
Survey Item M enaree Disagree Neutral Agree alee 
Tablet log-in and navigation in the app were easy. 4.00 - - 1 2 4 
How to administer the LRA and find test score results 4.38 ; ; , 1 5 
was Clear. 
It would be possible to devote 20 minutes per child (or 
per small group of children) for administering 4.38 - - 1 3 4 


Usefulness — Effectiveness of LRA Greenhouse at characterizing children’s literacy skills and learning needs 
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Table 7. LRA — Usefulness 


Strongly : Strongly 

Survey Item M Disagree Disagree Neutral Agree eres 
The LRA results appropriately differentiated children with 

: : a 4.38 - - 2 1 5 
different skill and ability levels. 
Children “at-risk” for reading difficulties were appropriately hte 7 é 
identified by the LRA. ; : : : 
| understood what the LRA results meant for my class. 4.33 - - - 4 2 


Fit — Appropriateness of LRA Greenhouse for preschool classrooms 
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Table 8. LRA and Greenhouse — Fit 


helping children develop. 


Survey Item n M eeongly Disagree Neutral Agree strongly 
Disagree Agree 

The LRA literacy items were developmentally appropriate. 8 4.63 - - - 3 5 

The LRA working memory task was developmentally 3 4.63 ; ; ; 3 5 

appropriate. 

The LRA behavior rating scale items were developmentally 3 475 ' ; : 5 6 

appropriate. 

Using information from the LRA to check on how children 

in my class are doing “fit” with my typical approach to 6 4.75 - - - 2 6 
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Facilitation — Capacity of LRA Greenhouse to support evidence-informed practices and assessment-guided instructional decision- 


making 


Table 9. LRA Greenhouse — Facilitation 


new ideas about my typical classroom practices. 


Strongly : Strongly 

Survey Item M Dune Disagree Neutral Agree Gree 
Knowing about assessment results improved my 450 : : : 3 3 
effectiveness at helping my class learn. , 
Knowing about “High Priority Needs” in my classroom 
helped me identify the children who most needed my 4.33 - - 1 2 3 
support for developing emergent literacy. 
The LRA Greenhouse activities helped me plan and 
implement appropriate activities to meet the diverse 4.75 - - - 2 6 
learning needs of children in my class. 
Using the Activities Checklist to monitor children’s learning 
positively impacted how | supported development in my 4.00 - - 2 2 2 
class. 
Implementing LRA Greenhouse activities helped me learn 

; : : 4.33 - - 1 2 3 
something new about the children in my class. 
Implementing LRA Greenhouse activities gave me insight or 4.50 ; - 1 1 A 


Impact on Practice — |mpact of LRA Greenhouse on how teachers develop children’s emergent literacy skills 
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Table 10. LRA Greenhouse — Impact on Practice 


Strongly P Strongly 

Survey Item M Diedaree Disagree Neutral Agree eree 
| am better able to identify children “at risk” of learning 413 ; ; 5 3 3 
difficulties now. 
| monitor children’s progress more carefully now. 4.38 - - 1 3 4 
| can discuss with parents the needs of their child differently 425 : ; 1 4 3 
now. 
| can better explain to other educators how to best support 

: ; : 4.13 - - 2 3 3 
the learning needs of children in my class now. 
| have changed the manner in which | work with children 425 : ; 5 5 4 


identified as having “High Priority Needs” now. 


